Using Person-Specific Muscle Fatigue Characteristics to Optimally Allocate Control in a Hybrid Exoskeleton – Preliminary Results

Abstract

Currently controllers that dynamically modulate functional electrical stimulation (FES) and a powered exoskeleton at the same time during standing-up movements are largely unavailable. In this paper, an optimal shared control of FES and a powered exoskeleton is designed to perform sitting to standing (STS) movements with a hybrid exoskeleton. A hierarchical control design is proposed to overcome the difficulties associated with developing an optimal real-time solution for the highly nonlinear and uncertain STS control model with multiple degrees of freedom. A higher-level robust nonlinear control design is derived to exponentially track a time-invariant desired STS movement profile. Then, a lower-level optimal control allocator is designed to distribute control between FES and the knee electric motors. The allocator uses a person’s muscle fatigue and recovery dynamics to determine an optimal ratio between the FES-elicited knee torque and the exoskeleton assist. Experiments were performed on human participants, two persons without disability and one person with spinal cord injury (SCI), to validate the feedback controller and the optimal torque allocator. The muscles of the participant with SCI did not actively contract to FES, so he was only tested with the powered exoskeleton controller. The experimental results show that the proposed hierarchical control design is a promising method to effect shared control in a hybrid exoskeleton.

I. Introduction

Restoring standing and walking mobility after lower-limb paralysis due to a spinal cord injury (SCI) is one of the top desired rehabilitation outcomes [1]–[6]. As a step towards this goal, the paper focuses on enabling persons with SCI to achieve sitting-to-standing (STS) movements with a hybrid exoskeleton. Research efforts focusing on these movements are of significance due to the fact that these movements precede standing or walking activities.

To rehabilitate persons with SCI, functional electrical stimulation (FES) is often prescribed. FES is a technique that uses external electrical currents to obtain desired muscle contractions in paralyzed muscles [2], [5], [7]–[16]. While FES control methods to perform STS movements exist [1], [2], [17], in some situations the sole use of FES may not generate sufficient torques to achieve the standing-up motion. These situations can arise in cases where muscles are too weak due to muscle atrophy after SCI, or the rapid onset of FES-induced muscle fatigue hinders sustained limb movements.

Recently, hybrid exoskeletons that combine FES and a powered exoskeleton have been proposed as rehabilitation technology to help people with SCI to regain standing and walking functions [6], [18], [19]. In a hybrid exoskeleton, FES-elicited torques can be supplemented with assistance from electric motors [6], [18]–[22]. By working in tandem, FES and the powered exoskeleton can potentially provide a reliable and consistent performance despite FES-induced fatigue or atrophied muscles. Additionally, using FES-elicited torque in conjunction with the powered exoskeleton may conserve power consumed by exoskeleton actuators [23]. This is potentially useful for prolonging the battery life of the exoskeleton while providing therapeutic benefits associated with FES.

In recent research, control methods for hybrid exoskeletons have been designed for performing a range of lower-limb activities [6], [18], [19], [23], [24]. In [24], both FES and a powered exoskeleton were used to achieve knee extension with an adaptive gain-based controller. The electric motors were controlled by a proportional derivative controller while the FES input was computed by an adaptive gain that is proportional to the motor current input. In [18], a cooperative knee joint controller was proposed for controlling a hybrid exoskeleton. The controller was tested on participants without disability. In that device, a PID controller determined the stimulation to the quadriceps muscles and an iterative learning controller determined the stimulation to the hamstrings. The stimulation parameters were modified according to a fatigue estimator that measures a torque-time integral. In [19], a cooperative control approach coordinates hip and knee motors with FES, which was applied to the hamstrings and the quadriceps muscles. The motors were controlled using a high-bandwidth position feedback and FES was modified by a difference between an estimated muscle torque and a reference torque profile. Also, the controller would turn off the stimulation if, in five consecutive trials, FES is unable to generate torque that is over a one third of the previous torque output. This approach was validated on a participant with SCI. In another class of hybrid FES-motor systems, lower-limb cycling exercises were achieved using a switched controller [22] and an iterative learning controller [25].

The rapid onset of FES-induced muscle fatigue is a major challenge. Thus, incorporating muscle fatigue and recovery dynamics into the shared control design can make it more effective. This type of dynamic shared control can also help maximize the potential benefits of using a hybrid exoskeleton such as alleviating muscle fatigue while reducing power consumption. However, hybrid exoskeleton controllers that dynamically share control between FES and the powered exoskeleton do not have the ability to support for standing and walking activities. A muscle fatigue dynamic model was adopted in a model predictive control (MPC) method to optimally allocate FES and an electric motor during seated knee extension [23]. A similar muscle fatigue model was used in a muscle-synergy inspired control scheme to coordinate FES and the powered exoskeleton during walking. The experiments were performed on a person who has SCI and participants without disability [26], [27]. The controller, however, did not dynamically optimize the shared control, and instead used time-invariant fixed synergies between FES and the powered exoskeleton.

Control allocation approaches in a hybrid device during a standing-up motion have been minimally reported. In [28], an FES-only controller governed by a PID-based inner loop and a virtual reference feedback tuning-based outer loop was designed to achieve standing and sitting motions with a person with paraplegia. Similarly, in [1], [2], only FES was used to achieve a standing-up motion, where the stimulation current amplitude was determined by minimizing the arm effort needed during the movement. In [29], a device with a pre-programmed FES was validated on persons with SCI. In [30], a timing to trigger FES was optimized to minimize the arm effort. In [4], [5], only a powered exoskeleton was used to assist a user during the standing-up motion. Recently, in [31] combined a powered exoskeleton and FES were used cooperatively to drive the motion to follow a user-controlled trajectory. However, stimulation amplitudes were set to be constant. In a more recent work [6], a PID controlled exoskeleton and event-triggered FES was used to achieve the standing-up motion. The studies mentioned so far have used pre-programmed FES or triggered constant FES inputs during the standing-up motion. These methods cannot automatically modulate FES and exoskeleton assistance based on the onset of muscle fatigue. An allocation based on the muscle fatigue prediction is important as it may optimize the exoskeleton and FES usage. This type of optimal allocation can potentially delay the onset of muscle fatigue, maximize the active exercise of the muscle, and lower power consumption of the exoskeleton.

In this paper a control scheme that can optimally allocate FES and the knee electric motors of the exoskeleton during the standing-up motion is proposed. The scheme uses a person-specific muscle fatigue model. This work builds on our previous work in MPC-based dynamic allocation of a hybrid neuroprosthesis [23], [32] that is capable only for seated knee extension. As the standing-up motion is more complex, new innovations were made in this paper to extend the MPC method. A higher-level robust feedback controller is derived to provide stabilizing total torques for both knee and hip joints. The controller tracks a desired standing-up motion that is governed by a virtual constraint [33], [34]. The constraint approach aids in the design of time-invariant joint trajectories that are coupled to a single monotonically increasing state-dependent function. This helps to avoid the design of multiple independent time-based trajectories for the lower-limb joints. At a lower-level, an online optimal control method is used to allocate the knee torque, which is computed by the higher-level controller, between the FES-induced torque and the electric motor torque at the knee joint. The optimization uses a normalized fatigue level that is predicted by a dynamic muscle fatigue model to determine the actuator allocation. Experiments were performed on two participants with no disabilities and one participant with SCI to validate the proposed control method.

II. Hybrid Exoskeleton System Dynamics and Control

A. System Dynamics

The combined exoskeleton and user dynamics during the standing-up motion, shown in Fig. 1, are modeled as

$$M(\mathit{q})\ddot{\mathit{q}}+C_m(\mathit{q},\dot{\mathit{q}})\dot{\mathit{q}}+G(\mathit{q})+F(\mathit{q},\dot{\mathit{q}})+\mathit{\omega }=T$$ (1)

where the terms $\mathit{q},\dot{\mathit{q}},\ddot{\mathit{q}}\in {ℝ}^2$ are the joint angular position vector, angular velocity vector, and angular acceleration vector, respectively. The vector q = [ q₁ q₂]^T contains the knee joint angular position, q₁ and the hip joint angular position, q₂. The matrix $M(\mathit{q})\in {ℝ}^{2\times 2}$ denotes the inertia matrix, $C_m(\mathit{q},\dot{\mathit{q}})\in {ℝ}^{2\times 2}$ denotes the Centripetal-Coriolis matrix, $G(\mathit{q})\in {ℝ}^2$ denotes the gravity vector, $F(\mathit{q},\dot{\mathit{q}})\in {ℝ}^2$ denotes the passive viscoelastic vector, due to viscoelastic tissue effects of the user and damping effects of the exoskeleton, and $\mathit{\omega }\in {ℝ}^2$ represents the unmodeled disturbances. The total torque vector

$$\mathit{T}\equiv {\left[\begin{array}{ll}{T}_1\hfill & T_2\hfill \end{array}\right]}^T={\left[\begin{array}{ll}{\tau }_k\hfill & {\tau }_h\hfill \end{array}\right]}^T+{\left[\begin{array}{ll}{\tau }_a\hfill & 0\hfill \end{array}\right]}^T$$ (2)

contains the knee motor torque ${\tau }_k\in ℝ$, hip motor torque ${\tau }_h\in ℝ$, and the FES-induced torque at the knee joint ${\tau }_a\left(q_1,{\dot{q}}_1,\mu ,u_{fes}\right)\in ℝ$, where $u_{fes}\in ℝ$ is the normalized FES current level, and $\mu \in ℝ$ the muscle fatigue [1], [35]. In (1) only the quadriceps muscles are stimulated by FES. The equation of motion (1) can be rewritten as

$$\ddot{\mathit{q}}+\mathrm{\Psi }(\mathit{q},\dot{\mathit{q}})+\mathit{v}=B(\mathit{q})\mathit{T}$$ (3)

where $\mathrm{\Psi }(\mathit{q},\dot{\mathit{q}})=M^{-1}C+M^{-1}F+M^{-1}G$, B (q) = M^-−1, v = M⁻¹ω. The model in (3) has the following properties and assumptions:

Fig. 1:

A figure depicting the hybrid exoskeleton for standing-up motion.

Property 1: The inertia matrix is symmetric, positive definite, bounded, and invertible [36]. Property 2: The dynamics in (1) follows the skew symmetric property [36], i.e., Ṁ − 2C_m = 0. Assumption 1: The human-machine standing-up motion can be described by a two active degree of freedom dynamics. Additional motion and torques (e.g., arms and ankle) are considered as disturbances. Assumption 2: The disturbance term $\mathit{\omega }\in {ℝ}^2$ is bounded. It contains unmodeled terms such as moments due to a user’s upper-body effort, spasticity, etc.

B. Standing-up Trajectory Planning

The reference joint angle vector that is defined as ${\mathit{q}}_d(\varsigma )\triangleq {\left[q_{d_1}{q}_{d_2}\right]}^T$ is computed using a virtual constraint approach [33]. The approach allows q_d to be time-invariant because it is a function of a state-dependent function $\varsigma (\mathit{q},\dot{\mathit{q}})$. Note that the vector is different from a time-dependent reference trajectory; e.g., q_d(t). We use Bezier polynomials to obtain the following reference output

$${\mathit{q}}_d=\left[\begin{array}{l}{b}_{1}o\varsigma (\mathit{q},\dot{\mathit{q}})\hfill \\ b_{2}o\varsigma (\mathit{q},\dot{\mathit{q}})\hfill \end{array}\right]$$ (4)

where

$$b_i(w)=\sum _{k=0}^M{\varrho }_k^i\frac{N_b!}{k!\left(N_b-k\right)!}{\varsigma }^k{(1-\varsigma )}^{M-k}.$$ (5)

In (5), $M,N_b\in {\mathbb{I}}^{+}$ denote the order and number of Bezier polynomial terms, respectively, ${\varrho }_k^i$ is a parameter found through a dynamic optimization, and $\varsigma (\mathit{q},\dot{\mathit{q}})$ is obtained according to

$$\varsigma =\frac{\theta (\mathit{q},\dot{\mathit{q}})-{\theta }^{+}}{{\theta }^{-}-{\theta }^{+}}$$ (6)

where θ⁻ and θ⁺are lower and upper limitations of the $\theta (\mathit{q},\dot{\mathit{q}})$, respectively [34]. The independent joint angle function, $\theta (\mathit{q},\dot{\mathit{q}})\in ℝ$ is defined as

$$\theta \triangleq {\zeta }_1{q}_1+{\zeta }_2{\dot{q}}_1+{\zeta }_3{q}_2+{\zeta }_4{\dot{q}}_2$$

where $\forall i=1,2,3,4{\zeta }_i\in ℝ$ are chosen such that $\theta (\mathit{q},\dot{\mathit{q}})$ is monotonically increasing.

By designing joint trajectories as in (4), the reference output becomes time-invariant. Using the reference joint angle vector, the reference joint angular velocity, ${\dot{\mathit{q}}}_d\triangleq {\left[{\dot{q}}_{d_1}{\dot{q}}_{d_2}\right]}^T$, and acceleration, ${\ddot{\mathit{q}}}_d\triangleq {\left[{\ddot{q}}_{d_1}{\ddot{q}}_{d_2}\right]}^T$ can be obtained.

C. Feedback Controller

It is desired to design a feedback controller, T = K(h_d, h), that stabilizes the standing-up motion, where ${\mathit{h}}_d\triangleq {\left[q_{d_1},{\dot{q}}_{d_1},{\ddot{q}}_{d_1},q_{d_2},{\dot{q}}_{d_2},{\ddot{q}}_{d_2}\right]}^T$ is defined as the reference vector and $\mathit{h}={\left[\begin{array}{l}{q}_1,{\dot{q}}_1,q_2,{\dot{q}}_2\hfill \end{array}\right]}^T$ is defined as the actual state vector. The control objective is to minimize the tracking error term $\mathit{e}\in {ℝ}^2$ that is defined as

$$\mathit{e}\triangleq {\mathit{q}}_d-\mathit{q}.$$

To aid control development an auxiliary signal, $\mathit{s}={\left[\begin{array}{ll}{s}_1\hfill & s_2\hfill \end{array}\right]}^T\in {ℝ}^2$, is defined as

$$\mathit{s}=\dot{\mathit{e}}+\lambda \mathit{e}.$$ (7)

where $\lambda \in ℝ$ is a constant. After taking the time derivative of (7) and on substituting (3),

$$\dot{\mathit{s}}={\ddot{\mathit{q}}}_d+\lambda {\dot{\mathit{q}}}_d-\lambda \dot{\mathit{q}}+\mathrm{\Psi }+\mathit{v}-B\mathit{T}.$$ (8)

Based on the subsequent stability analysis (see Appendix), a feedback law, T = K(h_d, h), is designed such that

$$\begin{gathered} K\left({\mathit{h}}_d,\mathit{h}\right)=\widehat{M}\left({\ddot{\mathit{q}}}_d+\lambda {\dot{\mathit{q}}}_d-\lambda \dot{\mathit{q}}+{\kappa }_2\mathit{s}\right)+{\widehat{C}}_m\mathit{s}+\widehat{C} \\ +\widehat{F}+\widehat{G}+{\kappa }_1\mathrm{sgn}(\mathit{s})+{\kappa }_0(\mathit{e},\dot{\mathit{e}})\mathit{s}+\mathrm{\Gamma }\mathrm{sgn}\left({\dot{\mathit{q}}}^T\mathit{s}\right)\dot{\mathit{q}}+\widehat{\mathit{\omega }} \end{gathered}$$ (9)

where ${\kappa }_1\in {ℝ}^{+}$, ${\kappa }_2\in {ℝ}^{+}$ are control gains, κ₀ is a positive and monotonically increasing function, $\mathrm{\Gamma }\in {ℝ}^{2\times 2}$ is a positive definite control gain matrix, $\widehat{M}\in {ℝ}^{2\times 2}$, ${\widehat{C}}_m\in {ℝ}^{2\times 2}$, $\widehat{C}\in {ℝ}^2$, $\widehat{F}\in {ℝ}^2$, $\widehat{G}\in {ℝ}^2$, $\widehat{\omega }\in {ℝ}^2$ are estimates of M, C_m, C, F, G, ω, respectively, where $C(\mathit{q},\dot{\mathit{q}})=C_m(\mathit{q},\dot{\mathit{q}})\dot{\mathit{q}}$ [36].

III. Model-based Optimal Allocation of the Actuators

The control allocation problem is to distribute T₁ in (2) between the FES generated knee torque, τ_a, and the knee electric motor, τ_k. Further, we are interested in computing the stimulation current amplitude for the quadriceps muscles that generates τ_a. The following model is used to determine the stimulation current amplitude.

A. Muscle Force Generation and Fatigue Model

The active knee torque, ${\tau }_a={\mathrm{\Phi }}_a\left(\varphi \left(q_1,{\dot{q}}_1\right),\mu ,u_{fes}\right):ℝ\times [0,1]\times [0,1]\in {ℝ}^{+}\cup \left\{0\right\}$ is

$${\tau }_a=\varphi \left(q_1,{\dot{q}}_1\right)\mu u_{fes},$$ (10)

where $\varphi \left(q_1,{\dot{q}}_1\right)=\left(c_2{q}_1^2+c_1{q}_1+c_0\right)\left(1-c_3{\dot{q}}_1\right)$ (c_i∀_i = 0, 1, 2, 3 are model parameters) is the torque-knee angle and knee angular velocity relationships, u_fes is the normalized stimulation amplitude, and μ is the normalized fatigue variable driven by the fatigue dynamics $\dot{\mu }={\mathrm{\Phi }}_{\mu }\left(\mu ,u_{fes}\right):[0,1]\times [0,1]\in {ℝ}^{+}\cup \left\{0\right\}$, which is

$$\dot{\mu }=\frac{\left({\mu }_{min}-\mu \right)u_{fes}}{T_f}-\frac{(1-\mu )\left(1-u_{fes}\right)}{T_r}$$ (11)

where $T_f\in {ℝ}^{+}$ is the fatigue time constant and $T_r\in {ℝ}^{+}$ is the recovery time constant. [23], [37]

B. Optimizer

An optimizer is designed to allocate between the electric motor and FES. The MPC controller considers the fatigue level trajectory over a time horizon, which helps to determine an optimal FES input. The optimization problem is stated as

$$\underset{{\overline{u}}_{fes}}{\min }J\left(t_k\right)={\int }_{t_k}^{t_k+T_N}\left\{{\overline{\tau }}_k^2+\frac{w}{\overline{\mu }+ϵ}{\overline{\tau }}_a^2+w_o{\left(\overline{\mu }-{\overline{\mu }}_o\right)}^2\right\}dt$$ (12)

$$\begin{gathered} \text{s.t.}{\overline{\tau }}_k+{\overline{\tau }}_a={\overline{T}}_1 \\ \overline{\mathit{T}}=K\left({\mathit{h}}_d,\overline{\mathit{h}}\right) \\ \widehat{M}(\overline{\mathit{q}})\ddot{\overline{\mathit{q}}}(t)+{\widehat{C}}_m(\overline{\mathit{q}},\dot{\overline{\mathit{q}}})\dot{\overline{\mathit{q}}}(t)+\widehat{F}(\overline{\mathit{q}},\dot{\overline{\mathit{q}}})+\widehat{G}(\overline{\mathit{q}})=\overline{\mathit{T}} \\ {\overline{\tau }}_a={\mathrm{\Phi }}_a\left(\varphi \left(q_1,{\dot{q}}_1\right),\overline{\mu },{\overline{u}}_{fes}\right) \\ \dot{\overline{\mu }}={\mathrm{\Phi }}_{\mu }\left(\overline{\mu },{\overline{u}}_{fes}\right) \\ {\overline{u}}_{fes}\in {\mathcal{U}}_{fes} \end{gathered}$$

where the objective index $J(t)\in {ℝ}^{+}\cup \left\{0\right\}$ in (12) is the cost function, ${\overline{\mu }}_o$ is the estimated fatigue of contralateral leg, where $\overline{\cdot }$ represents the nominal/estimated signals in the MPC, ${\mathcal{U}}_{fes}$ is the input constraint (normalized as [0, 1] [23], [38]), ϵ > 0 is a constant, and w, w_o > 0 are user-defined weights. When the optimal solution, $u_{fes}^{*}\left(t|:t\in \left[t_k,t_k+T_N\right]\right)=\mathrm{argmin}\{J(t)\}$, is found, $u_{fes}=u_{fes}^{*}\left(t|:t=t_k\to t_k+\epsilon \right)$ is applied to the system, where ε is an infinitesimal time constant that makes t_k+1 = t_k + ε [39]. For the ratio allocation optimization, a gradient search algorithm was adopted [39] to solve the optimization problem. The detailed algorithm can be found in Table III of the Appendix.

IV. Experiments and Results

Experiments were run with two participants without disability: Subject 1 (Sex: male; Age: 25; Weight: 60 kg; Height: 173 cm), Subject 2 (Sex: male; Age: 25; Weight: 65 kg; Height: 165 cm) and one participant with SCI: Subject 3 (Sex: male; Age: 51; Weight: 70.3 kg; Injury: T11). Approval for the study was obtained from the Institutional Review Board of the University of Pittsburgh.

The hybrid exoskeleton is shown in Fig. 2. In the device, two EC motors (Maxon Motors, Switzerland) were mounted on the knees, two gearmotors (LPA-17–100, Harmonic Drive, Japan) were attached to the hip joints. Inbuilt angular encoders in the motors were used to measure the hip and knee angles. An FES stimulator (RehaStim, Hasomed LLC, Germany) was used to transcutaneously stimulate the quadriceps muscles. In the experiments a constant frequency train, delivered at a frequency of 35 Hz, was used. A constant pulse width of 400 μs and a current modulating FES protocol was used. The stimulation current was modulated by the FES control input from the optimizer, as shown in Fig. 2. A real-time target machine (SpeedGoat, Switzerland) was used to run the SIMULINK^® control program and collect data in real-time. A hand-walker was used to help the user to keep the balance, but the participants were asked to not provide forces for lifting the bodies up.

Fig. 2:

This figure demonstrates the control loop of the proposed ratio allocation approach. The feedback controller provides stabilizing torques for both knee and hip joint. The hip torque signal, T₂, directly goes to the hip motor, while the knee torque signal, T₁, goes to an optimizer. The optimizer determines the amount of FES and the electrical motor torque by solving a user-defined cost function The computed FES-induced torque and the electric motor torque are constrained to be equal to the stabilizing knee torque, T₁.

Before running the STS experiments, a set of trials were conducted on each participant to estimate the model. The muscular model parameters of the participants, like c_i and d_i, were identified with the procedures reported in our previous works [40], and the rigid parts of the human model such as, segment length, center of mass, moment of inertia, etc., were estimated based on measurements and anthropometry. The fatigue model parameters were estimated by measuring the knee torque with a load cell while applying FES at its saturation level, specific to a subject’s muscles. After a rest period of ten seconds, FES with 0.5s pulse trains at the saturation level were used every 10s to measure the recovery rate of the muscles. Curves were fitted to the decreasing trend of normalized torques that reflect the onset of muscle fatigue and the increasing trend of torques that reflect muscle recovery. Fig. 3 shows an example of the fatigue parameters estimation. More details on fatigue model parameter estimation can be seen in [23], [40]. It is worthwhile to mention that the parameters of both knees of one person can be asymmetric. The participant with SCI did not respond to FES and the system identification could not proceed with this subject. During the experiments, participants were told to relax and avoid providing any voluntary force.

Fig. 3:

The figures show the fatigue and recovery parameter estimation, where the solid lines are the normalized forces and the dashed lines are the fitting curve. The upper panel demonstrates the estimation of T_f and the lower panel demonstrates the estimation of T_r. Parameters of subject 1 (S1) and subject 2 (S2) are also given.

The developed control framework was validated for STS tasks. A common desired time-invariant STS profile was designed for the joints of both legs. The reference path, which is generated by (4), depends on the measured angle of the right knee. After the transition, the standing position was held for up to 20 seconds to further validate the controller stability. The controllers were implemented separately for each leg but used the common virtual constraint function to maintain coordination between the legs. The control schematic is depicted in Fig. 2.

The control performance of Subject 1 is shown in Fig. 5 and Subject 3 in Fig. 6. Fig. 4 demonstrates the process of the standing-up motion achieved by Subject 1 and 3. The root mean square (RMS) of the joints performance, and average fatigue level of all experiments are summarized in Table. I. In all experiments, the participants finished the standing-up motion within 10s. The weights on the fatigue variable were varied to see their affects on muscle fatigue. According to the table, as the weight on FES is increased, the average fatigue variable also increases; i.e., muscle fatigue reduces. After the functionality of this hybrid device was verified, we tested the performance on S3. The control performance can be seen in Table. II. We also recorded the torque-time integration (TTI) of the knee motor torque during the experiments. The TTI indices are shown in Fig. 7. The result suggests that the hybrid device consumes less motor energy than the purely powered exoskeleton.

Fig. 5:

This figure shows the control performance of the knees (a) and hips (b), change of the fatigue level (c), and control inputs to the knee motors (d), hip motors (e), and FES (f) for a participant without disability.

Fig. 6:

This figure shows the control performance of the knees (a) and hips, (b), and control input of the knee motors (c) and hip motors (d) for a participant with SCI.

Fig. 4:

The figure shows STS motion of a participant without disability (a) and a participant with SCI (b) using the hybrid exoskeleton.

TABLE I:

This table summarizes the experimental data for the two participants without disability, where * is the representative shown in Fig. 5.

Participants without disability: Experimental Performance Data Summary During 0s To 10s the knee and hip joints RMS at steady state during 5s to 10s are shown in brackets
Subject 1		FES + Powered Exoskeleton
Trial No.		Knee RMS [Deg.]		Hip RMS [Deg.]		Average Fatigue		Weights
		Left	Right	Left	Right	Left	Right	w	wo
1		17.5 (3.0)	13.3 (1.3)	2.7 (0.5)	2.5 (0.3)	0.931	0.937	1	1
2	*	10.0 (0.8)	8.1 (1.2)	1.9 (0.2)	1.9 (0.2)	0.930	0.927	1	1
3		23.3 (4.7)	18.5 (4.7)	2.8 (1.1)	2.7 (0.9)	0.932	0.937	1	1
4		14.2 (1.4)	12.8 (3.4)	2.2 (0.2)	2.1 (0.3)	0.931	0.929	1	1
5		21.7 (2.2)	17.3 (2.1)	2.9 (0.4)	2.9 (0.4)	0.930	0.934	1	1
Average		17.3 (2.4)	14.0 (2.5)	2.5 (0.4)	2.4 (0.4)
Std.		5.4 (1.5)	4.1 (1.4)	0.4 (0.3)	0.3 (0.2)
		Powered Exoskeleton
		19.23 (3.4)	18.0 (6.1)	3.1 (1.5)	2.7 (1.2)
Subject 2		FES + Powered Exoskeleton
1		16.7 (0.6)	16.3 (2.2)	2.0 (0.2)	2.5 (0.3)	0.933	0.931	4	4
2		15.8 (0.2)	15.5 (2.2)	1.8 (0.4)	2.4 (0.1)	0.935	0.932	4	4
3		9.5 (1.5)	9.2 (0.3)	1.9 (0.2)	2.1 (0.5)	0.949	0.950	6	4
4		4.2 (2.1)	3.3 (0.7)	1.7 (0.5)	2.0 (1.1)	0.958	0.958	8	4
5		18.6 (1.3)	18.5 (3.9)	2.0 (0.2)	2.5 (0.3)	0.960	0.961	8	4
Average		12.9 (1.1)	12.6 (1.8)	0.0 (0.3)	0.0 (0.4)
Std.		5.9 (0.7)	6.2 (1.4)	0.0 (0.1)	0.0 (0.3)
		Powered Exoskeleton
		5.5 (1.5)	5.1 (0.3)	1.9 (0.3)	2.2 (0.2)

TABLE II:

This table summarizes the experimental data for the SCI participant, where * is the representative shown in Fig. 6.

Participants with SCI: Experimental Performance Data Summary During 0s To 10s the knee and hip joints RMS at steady state during 5s to 10s are shown in brackets
Subject 3
Trial No.		Knee RMS [Deg.]		Hip RMS [Deg.]
		Left	Right	Left	Right
1	*	7.1 (6.3)	3.5 (2.7)	2.6 (2.4)	2.9 (2.8)
2		6.1 (4.9)	4.0 (2.1)	2.8 (2.3)	2.8 (2.2)
Average		6.6 (5.6)	3.7 (2.4)	2.7 (2.3)	2.9 (2.5)
Std.		0.7 (0.9)	0.3 (0.4)	0.1 (0.0)	0.0 (0.4)

Fig. 7:

The figure shows the normalized TTI index for the hybrid exoskeleton and sole powered exoskeleton.

Due to the joint limitations of the participant with SCI, the terminal knee and hip joint angles were not set to 0° to avoid joint hyperextension. This can be seen in Fig. 4b where the trunk posture of the participant with SCI is not completely straight, unlike in an individual without disability. In Fig. 8 a representative result of the measured hand forces of the two subjects are shown, where both the hybrid case and motor-only case are considered for S1. It can be seen that the hand forces of the participants without disability go back to about 0N within 5s, but the hand forces in the case of the participant with SCI stay elevated. This suggests that the subject with SCI relied on the walker to balance himself during standing.

Fig. 8:

The figure shows the hand forces measured by the sensors during sensors for subject 1 (S1) and subject 3 (S3).

V. Discussion

A hybrid exoskeleton that combines FES and powered exoskeleton can provide reliable and consistent STS and walking performance despite FES-induced fatigue. This work builds on our previous work in MPC-based dynamic allocation of hybrid neuroprosthetic systems [23], [32], where only a knee extension motion was achieved. As STS is more complex, new innovations are required to apply the MPC method for dynamic allocation of FES and the electric motor inputs. The work in [23] used an MPC method for the first time to dynamically allocate FES and the electric motor inputs. This result was further extended to a tube-based MPC for a knee extension task that provides robustness to modeling uncertainties [32]. The results in this paper show how MPC algorithm can be built into a robust feedback controller and applied to the STS motion. This type of hierarchical control design is able to optimize the shared control between FES and the electric motors, while maintaining control robustness to modeling uncertainties.

The results also validate our earlier obtained results that show addition of FES can conserve electric motor torque (e.g. during knee extension [23] and standing-up [31]). Moreover, compared to [31] FES in this paper is shown to change dynamically. In [31], FES stimulation amplitude was kept constant. FES can be dynamically varied by using the optimizer, which can be tuned, specific to a person, by varying the cost function weights, w and w_o. For S1 we kept the weights the same and varied them for S2. We did this to see if the fatigue difference between two legs can be reduced, when w_o is increased. The results in Table I show that the difference decreased, but we do not know if the results are statistically significant. More investigation in to the effects of weight tuning of the optimizer will be considered in the future work.

Nonetheless, there are limitations of this paper. The preliminary TTI results do show the benefit of adding FES, i.e., it may be able to conserve electrical power consumption. However, additional data will be collected in the future to fully validate this claim. The TTI results were obtained from only 2 participants with no disabilities. The participant with SCI did not respond to FES. For this case, we could not compute the reduction in TTI as his muscles did not respond to FES. Despite this issue, the controller did facilitate STS in all the participants. It is worth noting the the hip extensors and knee extensors both work during the standing-up motion. This work only looked at stimulation of only the knee extensors but we plan to incorporate gluteal muscle stimulation in our future work. The use of fatigue model personalizes fatigue allocation based on the fatigue characteristics of a user, but additional set of experiments that identify the fatigue and recovery parameters are needed before the controller implementation. In the future, we plan to use a direct approach such as surface electromyography to measure/estimate the fatigue level instead of using a mathematical model of fatigue and recovery dynamics.

VI. Conclusion

This paper proposed a control scheme to optimally allocate FES and electric motors in a hybrid exoskeleton. In the scheme, a robust feedback controller provides the stabilizing control signals for the standing-up motion. The muscle fatigue and recovery model-based optimizer allocates the knee joint torque by determining an optimal ratio between the FES-induced torque and the motor torque. Experimental results validate the control performance and control allocation on participants without disabilities. The controller also enabled a participant with SCI to perform STS using the hybrid exoskeleton. Moreover, by using a predicted fatigue level, the controller can personalize the dynamic modulation of FES in a hybrid exoskeleton during the standing-up motion. More experiments are planned in the future to validate if the controller can reduce power consumption due to the addition of FES. Its advantages to reduce muscle fatigue and improving muscle health of persons with SCI will also be validated in future experiments.

Appendix

Control Error Dynamics and Stability Analysis

The control error dynamics due to the feedback controller in (9) and its stability analysis are provided below.

After multiplying (8) by M(q) and on substituting (9), the following equation is obtained

$$\begin{gathered} M\dot{\mathit{s}}=-C_m\mathit{s}+\tilde{M}\left({\ddot{\mathit{q}}}_d+\lambda {\dot{\mathit{q}}}_d\right)+\tilde{C}+\tilde{F} \\ +\tilde{G}+\tilde{\mathit{\omega }}-{\kappa }_1\mathrm{sgn}(\mathit{s}) \\ +{\kappa }_2\tilde{M}\mathit{s}+{\tilde{C}}_m\mathit{s}-{\kappa }_0\mathit{s} \\ -\mathrm{\Gamma }\mathrm{sgn}\left({\dot{\mathit{q}}}^T\mathit{s}\right)\dot{\mathit{q}}-\lambda \tilde{M}\dot{\mathit{q}}-{\kappa }_{2}M\mathit{s} \end{gathered}$$

where $\tilde{M}=M-\widehat{M}=\left[\begin{array}{cc}{\tilde{m}}_{11}& {\tilde{m}}_{12}\\ {\tilde{m}}_{21}& {\tilde{m}}_{22}\end{array}\right]$, ${\tilde{C}}_m=C_m-{\widehat{C}}_m$, $\tilde{C}=C-\widehat{C}$, $\tilde{F}=F-\widehat{F}$, and $\tilde{G}=G-\widehat{G}$, $\tilde{\mathit{\omega }}=\mathit{\omega }-\widehat{\mathit{\omega }}$. It can be further simplified to

$$\begin{gathered} M\dot{\mathit{s}}=-C_m\mathit{s}+\tilde{\mathit{n}}+\tilde{\mathit{y}}-{\kappa }_1\mathrm{sgn}(\mathit{s})-\lambda \tilde{M}\dot{\mathit{q}} \\ -\mathrm{\Gamma }\mathrm{sgn}\left({\dot{\mathit{q}}}^T\mathit{s}\right)\dot{\mathit{q}}+\Upsilon \mathit{s}-{\kappa }_0\mathit{s}-{\kappa }_{2}M\mathit{s}. \end{gathered}$$ (13)

In (13) $\Upsilon \in {ℝ}^{2\times 2}$ is defined as $\Upsilon \triangleq \left[\begin{array}{cc}{\epsilon }_1& {\epsilon }_2\\ {\epsilon }_3& {\epsilon }_4\end{array}\right]={\kappa }_2\tilde{M}+{\tilde{C}}_m$, where ${\epsilon }_i\in ℝ\forall i=1,2,3,4$, $\tilde{\mathit{y}}\in {ℝ}^2$ is defined as $\tilde{\mathit{y}}=\tilde{C}+\tilde{F}+\tilde{G}$, $\tilde{\mathit{n}}\in {ℝ}^2$ is defined as $\tilde{\mathit{n}}\triangleq {\left[{\tilde{n}}_1,{\tilde{n}}_2\right]}^T=\tilde{\mathit{\omega }}+\tilde{M}{\ddot{\mathit{q}}}_d+\lambda \tilde{M}{\dot{\mathit{q}}}_d$. It is also assumed that $\tilde{\mathit{y}}$ is bounded as [36]

$$\Vert \tilde{\mathit{y}}\Vert \le \tilde{Y}\left(\Vert \mathit{e},\dot{\mathit{e}}{\Vert }^2\right)$$

where $\tilde{Y}$ is a positive and monotonically increasing function.

Theorem 1. The control law in (9) makes the error dynamical system in (13) exponentially stable, provided the following gain conditions hold true

$$\begin{gathered} {\kappa }_0>max\left\{\left|{\epsilon }_i\right|;\forall i=1,2,3,4\right\}+\tilde{Y}(\mathit{e},\dot{\mathit{e}}) \\ {\kappa }_1>max\left\{\left|{\tilde{n}}_1\right|,\left|{\tilde{n}}_2\right|\right\} \\ \mathrm{\Gamma }=\left[\begin{array}{ll}{\gamma }_{11}\hfill & {\gamma }_{12}\hfill \\ {\gamma }_{21}\hfill & {\gamma }_{22}\hfill \end{array}\right]\forall i,j{\gamma }_{ij}\ge \lambda {\tilde{m}}_{ij}. \end{gathered}$$ (14)

Proof: Define a positive definite Lyapunov candidate, $V\in ℝ$

$$V=\frac{1}{2}{\mathit{s}}^{T}M\mathit{s}$$ (15)

such that λ_m ||s||² ≤ V ≤ λ_M ||s||², where λ_m and λ_M are the minimum and maximum eigenvalue of M, respectively.

The the time derivative of V is

$$\dot{V}={\mathit{s}}^{T}M\dot{\mathit{s}}+\frac{1}{2}{\mathit{s}}^T\dot{M}\mathit{s}.$$ (16)

On substituting (13) into (16), and by applying Property 2, we can obtain

$$\begin{gathered} \dot{V}=-{\kappa }_1{\mathit{s}}^T\mathrm{sgn}(\mathit{s})+{\mathit{s}}^T\tilde{n}+{\mathit{s}}^T\tilde{y}+{\mathit{s}}^T\Upsilon \mathit{s}-{\mathit{s}}^T{\kappa }_0\mathit{s} \\ -\lambda {\mathit{s}}^T\tilde{M}\dot{\mathit{q}}-{\mathit{s}}^T\mathrm{\Gamma }\mathrm{sgn}\left({\dot{\mathit{q}}}^T\mathit{s}\right)\dot{\mathit{q}}-{\mathit{s}}^T{\kappa }_{2}M\mathit{s} \\ \le -{\kappa }_1{\mathit{s}}^T\mathrm{sgn}(\mathit{s})+\left|{\mathit{s}}_1\right|\left|{\tilde{n}}_1\right|+\left|{\mathit{s}}_2\right|\left|{\tilde{n}}_2\right| \\ +\Vert \mathit{s}\Vert \left({\tilde{y}}_0+{\tilde{y}}_1\right)+{\mathit{s}}^T\Upsilon \mathit{s}-{\mathit{s}}^T{\kappa }_0\mathit{s} \\ -\lambda {\mathit{s}}^T\tilde{M}\dot{\mathit{q}}-{\mathit{s}}^T\mathrm{\Gamma }\mathrm{sgn}\left({\dot{\mathit{q}}}^T\mathit{s}\right)\dot{\mathit{q}}-{\kappa }_2{\mathit{s}}^{T}M\mathit{s}. \end{gathered}$$ (17)

By applying the control gain conditions (14), (17) can be further simplified to

$$\dot{V}<-{\kappa }_2{\mathit{s}}^{T}M\mathit{s}=-2{\kappa }_{2}V.$$

Also due to the gain condition (14), if $\Vert \mathit{e}(0),\dot{\mathit{e}}(0)\Vert \le \sqrt{{\tilde{Y}}^{-1}\left[{\kappa }_0-\max \left\{\left|{\epsilon }_i\right|;\forall i=1,2,3,4\right\}\right]}$

$$V(t)<V(0)\exp \left(-2{\kappa }_{2}t\right).$$

Therefore V(t) exponentially approaches zero with time, which means that s(t) exponentially approaches zero with time. ■

MPC Algorithm

The MPC algorithm can be found in the following table.

TABLE III:

Detailed MPC algorithm.

1	Initialization: j = 0
(1a)	Set the convergence tolerance εj.
(1b)	Measure $\mathit{q}\left(t_k\right),\dot{\mathit{q}}\left(t_k\right)$.
(1c)	Use virtual constraint and feedback controller to get ${\mathit{h}}_d(\tau ),\overline{\mathit{h}}(\tau )$, and T1(τ), where τ ∊ [tk, tk + TN].
(1d)	Choose initial control trajectory ${\overline{u}}_{fe\mathit{s}}(\tau )\in {\mathcal{U}}_{\left[t_k,t_k+T_N\right]}$ where τ ∊ [tk, tk + TN].
(1e)	Use ${\overline{u}}_{fe\mathit{s}}(\tau )$ and $\overline{\mathit{h}}(\tau )$ to obtain ${\overline{\tau }}_a(\tau )$, therefore, J(j)(tk) where τ ∊ [tk, tk + TN].
2	Optimal Solution Searching:
(2a)	Integrate backwards in time to solve for the costates l(j)(τ) by minimizing the Hamiltonian H = Jmpc + lTΦμ that $\dot{l}(\tau )=-\frac{\partial H\left(\overline{\mu },l,{\overline{u}}_{fe\mathit{s}}\right)}{\partial \overline{\mu }}$.
(2b)	Compute the search direction, a(j)(τ), from the Hamiltonian $a^{(j)}(\tau )=-\frac{\partial H\left(x,l,{\overline{u}}_{fe\mathit{s}}\right)}{\partial {\overline{u}}_{fe\mathit{s}}}$.
(2c)	Compute the optimal step size, σ(j), with adaptive setting in [39].
(2d)	Update the control trajectory
	${\overline{u}}_{fe\mathit{s}}^{(j+1)}(\tau )=\psi ({\overline{u}}_{fe\mathit{s}}^{(j)}+{\sigma }^{(j)}{a}^{(j)})$
	where ψ denotes the constraints.
(2e)	Use ${\overline{u}}_{fe\mathit{s}}^{(j+1)}$ to get J(j+l)(tk).
(2f)	Check Quit Conditions
	(i) quit if $\left|J^{(j+1)}\left(t_k\right)-J^{(j)}\left(t_k\right)\right|\le {\epsilon }_j$,
	(ii) quit if j has exceeded the max iteration limit, Nt,
	(iii) otherwise set j = j + 1 and reiterate gradient step from (2a).