An Optimized Proportional-Derivative Controller for the Human Upper Extremity with Gravity

Abstract

When Functional Electrical Stimulation (FES) is used to restore movement in subjects with spinal cord injury (SCI), muscle stimulation patterns should be selected to generate accurate and efficient movements. Ideally, the controller for such a neuroprosthesis will have the simplest architecture possible, to facilitate translation into a clinical setting. In this study, we used the simulated annealing algorithm to optimize two proportional-derivative (PD) feedback controller gain sets for a 3-dimensional arm model that includes musculoskeletal dynamics and has 5 degrees of freedom and 22 muscles, performing goal-oriented reaching movements. Controller gains were optimized by minimizing a weighted sum of position errors, orientation errors, and muscle activations. After optimization, gain performance was evaluated on the basis of accuracy and efficiency of reaching movements, along with three other benchmark gain sets not optimized for our system, on a large set of dynamic reaching movements for which the controllers had not been optimized, to test ability to generalize. Robustness in the presence of weakened muscles was also tested. The two optimized gain sets were found to have very similar performance to each other on all metrics, and to exhibit significantly better accuracy, compared with the three standard gain sets. All gain sets investigated used physiologically acceptable amounts of muscular activation. It was concluded that optimization can yield significant improvements in controller performance while still maintaining muscular efficiency, and that optimization should be considered as a strategy for future neuroprosthesis controller design.

1. Introduction

Spinal cord injury (SCI) impairs movement and sensation below the level of injury. High-level SCI, which affects the cervical C1 – C4 levels, compromises voluntary motor function below the neck. Although communication between the brain and peripheral neuromuscular system is impaired, muscle function remains intact. Functional Electrical Stimulation (FES) is a technology that uses electrical current to activate peripheral nerves that otherwise would be inactive due to injury (Crago et al., 1996) to restore useful muscular movement. FES neuroprostheses have been applied to numerous physiological systems, including upper extremity function, which is addressed in the present study.

Feedforward control is the form most commonly used for clinical FES applications (Peckham and Knutson, 2005; Lynch and Popovic, 2008). It entails calculating and applying muscle stimulation patterns using available information about the system, without the use of feedback signals. It is simple to implement and does not require sensors; however, this absence of sensors also makes the success of the movements generated heavily dependent on accurate models of the controlled system and environment.

Feedback control requires the use of sensors, which detect arm properties and allow the controller to correct its actions if they deviate from the desired behavior. Upper extremity (UE) FES applications of feedback control have included shoulder function (Yu et al., 2001), elbow extension (Giuffrida and Crago, 2001; Memberg et al., 2003), hand grasp (Kilgore et al., 1989), and wrist stabilization (Lemay and Crago, 1997).

Additionally, more advanced upper extremity FES controllers have been investigated. These can involve the combination of feedforward and feedback control (Abbas and Chizeck, 1995; Blana et al., 2009), reinforcement learning (Izawa et al., 2004; Thomas, 2009; Jagodnik, 2014), and artificial neural networks (Giuffrida and Crago, 2005; Hincapie and Kirsch, 2009).

Many projects that develop advanced controllers compare their new control method to more basic feedback control, e.g. proportional-derivative (PD) or proportional-integral-derivative (PID), to demonstrate the superiority of the newly-developed advanced controller. However, there is often minimal effort invested in adequately tuning the feedback controllers intended for comparison, and we hypothesize that these feedback controllers may often perform worse than they would have, had they been properly tuned. PID controller tuning algorithms include the Ziegler-Nichols method (Ziegler and Nichols, 1942; Astrom and Hagglund, 2004) and the Chien, Hrones, and Reswick method (Chien et al., 1952). However, these tuning methods can often result in poor performance (Astrom and Hagglund, 2001), particularly for nonlinear systems such as FES control. For example, when using Ziegler-Nichols tuning, overshoot is common for nonlinear systems (Dey and Mudi, 2009). Such tuning algorithms cannot be considered optimized. Because these simpler feedback controllers have not been given the same care in tuning as the more advanced controllers to which they are being compared, it is likely that inaccurate conclusions may be drawn when comparing these two classes of control algorithms.

For this reason, we propose to mathematically optimize a proportional-derivative (PD) controller gain set for a 3-dimensional human shoulder and arm system, and to compare its performance on dynamic reaching tasks to PD controller gain sets tuned using standard algorithms. We hypothesize that optimization will yield significantly improved performance when compared with standard, non-optimal, tuning methods. PD control was selected because it represents a basic feedback control architecture, and because goal-directed reaching movements with a single endpoint specified per task are being performed (Heaviside step function with no explicit trajectory specified); such a task specification could result in compromised performance should an integral control component be added, as in a PID controller. Additionally, PD control is consistent with the Equilibrium Point hypothesis, which effectively explains certain features of motor control (Bizzi et al., 1992; Feldman et al., 1998).

We have previously determined for a planar arm system that using simulated annealing to optimize PD control can yield excellent performance (Jagodnik and van den Bogert, 2010). To extend our previous work, we optimize a PD controller to perform goal-oriented reaching movements, using a 3-dimensional biomechanical model of a human arm that has 5 degrees of freedom (DOF). We explore two PD controller architectures: one with 2 gains, and another with 10. The optimized controller gain sets are applied to a large variety of point-to-point reaching tasks, and tested for their ability to generalize to tasks for which they had not been optimized, and for their ability to withstand muscular fatigue. The performance of our optimized controller gain sets is compared with that of three other PD controller gain sets that have not been optimized for this system, and conclusions are drawn about the utility of optimization for neuroprosthesis controller development.

2. Methods

2.1. Biomechanical Model

For all experiments described, a 3-dimensional (3D) computational musculoskeletal model of the human arm was used that has 5 degrees of freedom (DOF) (Table 1) and 102 muscle elements grouped into 22 muscles (Table 2) (Chadwick et al., 2009). The model includes gravity and uses a fixed scapula as the base of the model. All joints are modeled as hinges, with the glenohumeral joint consisting of three such hinges (Chadwick et al., 2009); this joint is modeled according to the Y-Z’-Y” convention (Fig. 1). Rotations and displacements are defined according to Wu et al. (2005). Muscles are modeled as a Hill structure (Zajac, 1988) with activation dynamics and contraction dynamics. Passive muscle force was not included because the difficulty in correctly estimating the L_slack value for the parallel elastic element (Chadwick et al., 2009) for all muscle elements was found to negatively impact model performance. The model was implemented for forward dynamic muscle-driven simulation.

Table 1.

Angular limits for 3D arm model degrees of freedom.

Degree of Freedom	Min Angle (°)	Max Angle (°)
Plane of elevation	−10	90
Angle of elevation	5	90
Internal rotation	−55	70
Elbow flexion	5	140
Forearm pronation	5	160

Table 2.

Muscles included in 3D arm model, with joints crossed by each muscle (GH: Gleno-humeral, HU: Humero-Ulnar; RU: Radio-Ulnar) and number of elements used to model each muscle.

Muscle	Joints Crossed	# of Elements
Deltoid, scapular part	GH	11
Deltoid, clavicular part	GH	4
Coracobrachialis	GH	3
Infraspinatus	GH	6
Teres minor	GH	3
Teres major	GH	4
Supraspinatus	GH	4
Subscapularis	GH	11
Biceps, long head	GH, HU, RU	1
Biceps, short head	GH, HU, RU	2
Triceps, long head	GH, RU	4
Latissimus dorsi	GH	6
Pectoralis major, thoracic part	GH	6
Pectoralis major, clavicular part	GH	2
Triceps, medial head	HU	5
Brachialis	HU	7
Brachioradialis	HU, RU	3
Pronator teres	HU, RU	2
Supinator	HU, RU	5
Pronator quadratus	RU	3
Triceps, lateral head	HU	5
Anconeus	HU	5

Fig. 1.

Joint angle definitions for 3D arm model. The shoulder joint is modeled according to the Y-Z’-Y” convention.

2.2. Controller Design and Optimization

The PD controller calculates muscle stimulation values proportional to joint angle errors and angular velocities (Eq. (1)). The sign of the moment arm is also included in this equation, to ensure production of movement in the proper direction, given that each muscle can potentially affect multiple DOFs. The PD controller initially calculates 102 outputs, one for each muscle element. For each of the 22 muscles, the calculated stimulation values of its N elements were averaged, and this mean value was applied to all elements of the muscle, resulting in a total of 22 unique muscle stimulation values applied per iteration (Fig. 2). This grouping constrained the system to a more realistic approximation of an FES system, in which an electrode would stimulate discrete muscles or muscle groups, rather than individual muscle elements.

Fig. 2.

System block diagram of arm with proportional-derivative (PD) control. The PD Controller block is specified by the PD controller equation (Eq. (2)). u() are muscle stimulation values, θ(5) are joint angles, and θ̇(5) are angular velocities, with the subscript “targ” indicating target values.

The PD controller equation is:

$$u_i(t)=\sum _{j=1}^5-\mathit{\text{sign}}(R_{ij}(\theta ))[(K{p}_j*({\theta }_{\mathit{\text{actual}}}-{\theta }_{t\text{arg}et})+K{d}_j*({\stackrel{•}{\theta }}_{\mathit{\text{actual}}}-{\stackrel{•}{\theta }}_{t\text{arg}et})]$$ (1)

where

$$\mathit{\text{sign}}(R_{ij}(\theta ))=\{\begin{array}{cc}\hfill 1,\mathit{\text{if}}& R(\theta )\ge 0.5mm\hfill \\ \hfill 0,\mathit{\text{if}}& |R(\theta )|<0.5mm\hfill \\ \hfill -1,\mathit{\text{if}}& R(\theta )\le -0.5mm\hfill \end{array}$$ (2)

and R_ij(θ) is moment arm, index i represents the muscle, index j represents degree of freedom, Kp_j is the proportional gain matrix about degree of freedom j, Kd_j is the derivative gain matrix about degree of freedom j, θ is joint angle, and $\stackrel{•}{\theta }$ is joint angular velocity. We assume that we do not need to know the actual moment arm, which will vary between people and when comparing human physiology to our model; it is assumed that the direction of action is the same as in our evaluated model.

Two versions of the PD controller were implemented: one using 2 gain parameters (a single proportional gain (K_p) and a single derivative gain (K_d)) and the other using 10 gain parameters, with a K_p and a K_d value for each of the 5 DOF sensors (Appendix, Table 1).

Appendix Table 1.

Gain definitions for the 10-parameter proportional-derivative (PD) controller gain set.

	DOF1	DOF2	DOF3	DOF4	DOF5
Kp	Kp1	Kp2	Kp3	Kp4	Kp5
Kd	Kd1	Kd2	Kd3	Kd4	Kd5

All simulations were implemented using the C programming language, and this work made use of the High Performance Computing Resource in the Core Facility for Advanced Research Computing at Case Western Reserve University.

The gain matrix, K, was optimized by the cost function equation (Eq. (3)), which consists of three terms: a position error term, measuring the error between endpoint hand position and specified target position, in cm; an orientation error term, measuring error between actual endpoint forearm orientation and specified target orientation, in degrees; and an effort term (f_effort, with values ranging from 0.0 (no activation) to 1.0 (full activation)) determined by the continuously-valued muscle activation level. Endpoint (i.e., end-of-movement) position and orientation errors were used to quantify performance.

The cost function is defined as:

$$f(K)=(W_1*f_{\mathit{\text{posErr}}})+(W_2*f_{\mathit{\text{orientErr}}})+(W_3*f_{\mathit{\text{effort}}})$$ (3)

where

$$f_{\mathit{\text{posErr}}}=\frac{1}{T{N}_m}\sum _{i=1}^{N_m}\left|p_i\right|$$ (4)

$$f_{\mathit{\text{orientErr}}}=\frac{1}{T{N}_m}\sum _{i=1}^{N_m}\left|o_i\right|$$ (5)

$$f_{\mathit{\text{effort}}}=\sqrt{\frac{1}{102T{N}_m}\sum _{i=1}^{N_m}\sum _{j=1}^{102}\underset{0}{\overset{T}{\int }}{(\mathit{\text{act}}(t))}^{2}dt}$$ (6)

where |p_i| is endpoint position error of movement i in cm; |o_i| is endpoint orientation error of movement i in degrees; and act is muscle activation (0.0 to 1.0). N_m = 100 is number of movements performed. T is duration of simulated movements, and was selected to be 2 seconds because able-bodied reaching movements (~0.5 s) should complete within this period (Gottlieb et al., 1997), while allowing additional time for muscle-weakened movements (as in the Fatigue Robustness Test, described below) to complete.

Gain matrices were optimized using the simulated annealing algorithm (Kirkpatrick et al., 1983; Goffe et al., 1994) over a set of 100 dynamic, goal-oriented reaching tasks. The temperature reduction factor reduced the “temperature” (i.e. tendency to explore the state space) parameter by a specified percentage at each time step; once this parameter reached a value <10⁻⁶, the optimization was terminated. A range of values for this parameter was tested from 1 × 10⁻¹⁰ to 0.5. Values of 1 × 10⁻⁷ and 1 × 10⁻⁴ were found to produce the best results in the 2- and 10-parameter controllers, respectively. Multiple random seed values were used for each optimization, to verify that global optimal solutions were being identified. The initial solution for the 2-parameter controller was taken from (Jagodnik and van den Bogert, 2010), and the initial solution for the 10-parameter controller was the optimized 2-parameter gain set.

The wrist was selected as the most distal point for our model; specifically, we used the midpoint of the ulnar and radial styloid processes. Our model outputs the transformation matrix describing the position and orientation at the endpoint. Position and orientation errors are calculated from the relative transformation between final posture and specified target posture.

Cost function weights W_x in Eq. (3) were selected as follows: W₁ = 1.0, W₂ = 1.0, W₃ = 80.0. These values balanced the cost function components so that no term would dominate the cost function calculation. The controller gains, K_p and K_d, were constrained between 0 and 2. A gain of 2 produces full muscle stimulation for 0.5 rad of joint angle error or 0.5 rad/sec of angular velocity error.

The reaching movement task sets used in our work were created in the following way. Each goal-oriented reaching task was specified by two values: an initial position and a target position, with each position described by one joint angle value for each of the 5 DOF. A set of 25 static positions (Appendix, Table 2) was selected that represents useful arm positions spanning the functional arm workspace (Gresham et al. 1986; Marino et al. 1998; Cornwell et al. 2012). From this set of 25 static positions, a set of 600 dynamic reaching movements was generated by each position being paired with every other position (excluding static tasks having identical initial and target positions). From this set, a subset of 100 tasks was randomly selected for controller optimization. The remaining 500 tasks were used for the post-optimization controller tests described below. The 100-task training set had mean position displacement (i.e., distance between the initial position and specified target position of the wrist) of 36.32 ± 13.65 cm (mean ± SD), and mean orientation displacement of 37.28 ± 27.87°.

Appendix Table 2.

Static arm positions used to generate training and testing task sets.

Task	Description	Angles (in °) by DOF
		DOF1	DOF2	DOF3	DOF4	DOF5
1	outstretched to table	80.0	65.0	30.0	10.0	70.0
2	touch side of face	60.0	89.0	−45.0	139.0	40.0
3	press elevator button	65.0	85.0	10.0	40.0	70.0
4	hold mug with a straw	40.0	80.0	−10.0	139.0	75.0
5	arm outside central workspace	1.0	85.0	30.0	50.0	75.0
6	armrest position	5.0	25.0	5.0	85.0	140.0
7	outstretched to countertop	89.0	89.0	30.0	5.0	75.0
8	outstretched to object at right	70.0	75.0	30.0	15.0	70.0
9	hand at mouth	65.0	75.0	−30.0	115.0	25.0
10	hand in front, in usable space #1	40.0	50.0	5.0	100.0	75.0
11	hand in front, in usable space #2	45.0	60.0	10.0	110.0	80.0
12	extend to side	5.0	85.0	35.0	10.0	80.0
13	hang near side, elbow slightly flexed	20.0	30.0	−10.0	30.0	90.0
14	wave hello	1.0	85.0	−55.0	90.0	100.0
15	arm in neutral position #1	10.0	50.0	10.0	75.0	85.0
16	arm in neutral position #2	20.0	55.0	15.0	70.0	80.0
17	elevate & extend forward	88.0	85.0	40.0	10.0	80.0
18	extend in 30° plane, elbow flexed	30.0	70.0	45.0	80.0	80.0
19	extend forward, elbow flexed	65.0	80.0	45.0	40.0	75.0
20	extend in 50° plane	50.0	75.0	30.0	15.0	75.0
21	extend to side, slightly forward	10.0	75.0	35.0	10.0	65.0
22	touch chest	60.0	50.0	−10.0	139.0	10.0
23	hand at table height	80.0	55.0	30.0	5.0	75.0
24	pull door open, elbow flexed	−10.0	85.0	30.0	90.0	40.0
25	extend in 45° plane, elbow flexed	45.0	85.0	30.0	55.0	75.0

DOF: Degrees of Freedom

2.3. Simulation Experiments

2.3.1. Effect of Controller Architecture

For this investigation, we optimized a PD controller with 2 gain parameters, and another with 10 gain parameters. Refer to Section 2.2 for details.

2.3.2. Generality and Fatigue Robustness Tests

The set of 500 dynamic, goal-oriented reaching movements (Section 2.2) was used to perform both the Generality Test and the Fatigue Robustness Test. This set had the following properties: mean position displacement of 32.97 ± 14.85 cm, mean orientation displacement of 34.62 ± 24.85°.

For the following controller tests, in addition to testing the two gain sets optimized on the 3D arm model, three other 2-parameter PD controller gain sets were tested (Table 3). The Hand-Tuned controller is a manually-tuned gain set whose K_d was initially set to 0, and whose K_p was increased until target overshoot and ringing were first observed; K_d was then increased until overshoot was eliminated. The Ziegler-Nichols gain set used this eponymous frequency-response tuning method (Ziegler and Nichols, 1942; Astrom and Hagglund, 1995). The 2D optimized 2-parameter gain set resulted from our previous work using simulated annealing to optimize a PD controller for a 2-DOF planar arm model (Jagodnik and van den Bogert, 2010).

Table 3.

The five tested controller gain sets.

# Gains	Description	Kp	Kd
		1.841	0.234
		1.875	0.172
10	3D optimized	1.994	0.124
		1.298	0.104
		1.551	0.038
2	3D optimized	1.797	0.114
2	2D optimized	0.962	0.128
2	3D Hand-Tuned	0.80	0.10
2	3D Ziegler-Nichols	0.468	0.05

2.3.2.1. Generality Test

The purpose of the Generality Test was to apply the gain sets to tasks for which they had not previously been optimized, and to observe the resulting performance. PD controllers using each of the 5 gain sets were applied to the 500-task testing set, and accuracy and effort data were collected. Kruskal-Wallis ANOVA analysis with multiple comparison was performed to compare the five PD controller gain sets for the following performance metrics: endpoint position error; endpoint forearm orientation error; and muscular effort measured as the mean muscle activation over the entire movement and over only the final timestep of the movement.

2.3.2.2. Fatigue Robustness Test

This test examined gain set robustness to weakening the muscles of the arm model. We applied 11 levels of fatigue to the muscles over each movement; all muscle elements were weakened by the same amount, such that the maximum force that each element could produce was decreased by the same percentage. Fatigue levels were selected by counting up by 5%, from 0% to 50% fatigue, with the upper limit approximating the estimated maximal levels of muscular fatigue observed in SCI clinical subjects, assuming that they are regularly being treated using electrical stimulation (Thomas et al., 1997b; Belanger et al., 2000). Mean and standard deviation values for each PD controller gain set performing at each of 11 fatigue levels were analyzed for position error, orientation error, and mean muscular effort metrics.

3. Results

The gains resulting from the optimizations performed in this study are presented (Table 3). This table also lists the additional three PD controller gain sets used for comparison analyses.

3.1. Generality Test

Position and orientation errors (Fig. 3) and endpoint and mean effort values (Fig. 4) for the PD controllers using all five gain sets on the Generality Test are presented. These values include the endpoint position errors on the Generality Test (Fig. 3A). Kruskal-Wallis ANOVA analysis showed a significant effect of gain set on position error (χ²(4) = 606.23, p ≪ 0.001). There were significant differences (p < 0.01) between all gain set pairwise comparisons for the endpoint position error metric, except for the comparison between the two gain sets optimized on the 3D arm model (2-parameter vs. 10-parameter gain sets had position errors of 3.0 ± 3.2 cm vs. 2.9 ± 3.1 cm, respectively). Animations of controllers specified by different gain sets controlling the arm for a set of representative tasks are available as Electronic Supplementary Material.

Fig. 3.

Boxplots showing (A) endpoint position errors and (B) endpoint orientation errors for the Generality Test. ‘3D 10-param’ is the set of 10 proportional-derivative (PD) gains optimized on the 3-dimensional arm model; ‘3D 2-param’ is the pair of 2 gains optimized on the 3-dimensional arm model; ‘2D 2-param’ is the pair of gains optimized on the planar arm model; ‘Hand-Tuned’ indicates the gain set manually tuned on the 3D arm model; and ‘Ziegler-Nichols’ denotes the gain set tuned on the 3D arm model using the Ziegler-Nichols method. Red ‘+’ symbols indicate individual outlier values.

Fig. 4.

Boxplots showing (A) endpoint muscular effort and (B) whole-task mean muscular effort values for the Generality Test. ‘3D 10-param’ is the set of 10 proportional-derivative (PD) gains optimized on the 3-dimensional arm model; ‘3D 2-param’ is the pair of 2 gains optimized on the 3-dimensional arm model; ‘2D 2-param’ is the pair of gains optimized on the planar arm model; ‘Hand-Tuned’ indicates the gain set manually tuned on the 3D arm model; and ‘Ziegler-Nichols’ denotes the gain set tuned on the 3D arm model using the Ziegler-Nichols method. Red ‘+’ symbols indicate individual outlier values.

Endpoint orientation errors for the Generality Test display an increasing trend across tuning methods (Fig. 3B), with the gain sets optimized on the 3D arm model having the smallest errors (4.3° ± 3.9° and 3.9° ± 3.2° for 2- and 10-parameter gain sets, respectively), and the Ziegler-Nichols gain set showing the largest errors (16.8° ± 10.2°). The Kruskal-Wallis test showed a significant effect of gain set on orientation error (χ²(4) = 816.68, p ≪ 0.001). Significant differences (p < 0.01) existed between all gain set pairs for this metric, excepting the comparison between the gain sets optimized on the 3D arm model.

Trends in the endpoint muscular effort values for the Generality Test were minimal (Fig. 4A). Means and distributions among the five gain sets were very similar (each gain set had values of approximately 0.06 ± 0.02), although the two optimized gain sets had several larger outlier values in the range of 0.23 – 0.30 that were absent in the other three gain sets. No significant differences existed between any pair of gain sets for this metric.

The whole-task mean effort values for the Generality Test tended to be higher for the optimized gain sets (Fig. 4B). The mean values of the two controllers optimized on the 3D arm model were 0.11 ± 0.03 and 0.10 ± 0.03 for the 2- and 10-parameter gain sets, respectively, while the optimized planar controller, manually-tuned and Ziegler-Nichols gain sets had mean values of 0.08 ± 0.02, 0.08 ± 0.02, and 0.07 ± 0.02, respectively. The Kruskal-Wallis test showed a significant effect of gain set on mean effort (χ²(4) = 709.72, p ≪ 0.001). Significant differences (p < 0.01) were observed between all gain set pairs for this metric, excepting the pair consisting of the manually-tuned and 2-parameter planar optimized gain sets.

3.1.1. Illustration of an Example Task from the Generality Test

To demonstrate the relative performance of PD controllers on a specific task, we selected a representative movement from the 500-task set (Fig. 5). The endpoint position error (Fig. 6A) and endpoint orientation error (Fig, 6B) for the PD controller are shown using all five gain sets. Consistent with the trends in Figs. 3 and 4, the two optimized gain sets had the smallest errors, while the Ziegler-Nichols-tuned gain set had substantially larger errors than all others. For all controllers, the hand position remains constant once the final hand position has been attained.

Fig. 5.

Example task visualization: front (A) and side (B) views of the initial position (Position description: “Hand in front, in usable space #1.”), and front (C) and side (D) views of the target position (Position description: “Outstretched to object at right.”).

Fig. 6.

(A) Endpoint position errors and (B) Endpoint orientation error for 5 proportional-derivative (PD) controller gain sets performing the example reaching task (Figure 5) from the Generality Test. ‘3D 10-param (opt)’ is the set of 10 PD gains optimized on the 3-dimensional arm model; ‘3D 2-param (opt)’ is the pair of 2 gains optimized on the 3-dimensional arm model; ‘2D 2-param (opt)’ is the pair of gains optimized on the planar arm model; ‘Hand-Tuned’ indicates the gain set manually tuned on the 3D arm model; and ‘Ziegler-Nichols’ denotes the gain set tuned on the 3D arm model using the Ziegler-Nichols method.

3.2. Fatigue Robustness Test

The results of the Fatigue Robustness Test show much higher errors for the Ziegler-Nichols gain set compared with the other gain sets (Fig. 7). The performance trends in this figure are consistent with those of the Generality Test (Figs. 3, 4, and 6).

Fig. 7.

Fatigue Robustness test results. Plots show performance metrics as a function of muscular fatigue over all muscles. Error bars show standard deviation of the estimated mean. (A) Endpoint position error in cm. (B) Endpoint orientation error in degrees. (C) Mean muscular effort over the entire movement.

Endpoint position error shows a consistent ordering of gain set behavior (Fig. 7A). The two 3D model-optimized gain sets had very similar position error values; even at 50% fatigue, this error remained <5 cm. Relative trends among the other three gain sets remained consistent with previous figures. Endpoint orientation error shows similar trends (Fig. 7B). In contrast, muscular effort across each complete movement (Fig. 7C) shows an opposite ordering of gain set properties, compared with the error values of Figs. 7A and 7B.

4. Discussion

In this study, we optimized a PD controller for a 3D model of the human arm using simulated annealing. Two different PD control architectures were investigated, with 2 and 10 gain parameters. When PD controllers using the optimized gain sets were applied to a large set of tasks on which the gains had not been optimized, to test ability to generalize, both gain sets achieved excellent accuracy, with the 10-parameter gain set slightly, but not significantly, outperforming the 2-parameter gain set on all performance metrics. When compared with three other gain sets that had not been optimized for the 3D arm, the two optimized gain sets had consistently smaller endpoint position and orientation errors than the other gain sets. The Ziegler-Nichols-tuned and manually-tuned gain sets required less muscular effort than the 3D-optimized gain sets, although muscular effort across all gain sets and tasks remained within a physiologically reasonable range.

When considering composite performance of the PD controller using each gain set, we found that our two optimized gain sets had significantly better accuracy while still requiring acceptably low levels of muscular activation.

The PD controllers using the two optimized gain sets had strongly similar characteristics. In the Generality Test, Kruskal-Wallis analysis revealed no significant differences between the PD controllers using the two optimized gain sets on the metrics of cost function, endpoint position error, endpoint orientation error, and endpoint muscular effort. Both controllers had good accuracy, with endpoint position and orientation errors averaging ~3 cm and ~4°, respectively. In the context of restoring arm movement for SCI rehabilitation, this level of accuracy is considered acceptable for achieving useful movement, with the expectation that fine adjustments can be made by the subject (e.g. Thomas et al., 1997a, b). Using the optimized 2-parameter gain set for future clinical applications may be warranted, as manually fine-tuning this controller will be much more tractable than modifying 10 parameters.

We had previously found for a planar arm system that selecting a PD controller with more gain parameters does not necessarily result in significantly improved performance (Jagodnik and van den Bogert, 2010). Similarly, in the present work, we find again that there may not be a functional benefit to using a controller with more parameters, when the performance of the simpler 2-parameter gain set is nearly identical to the more complex 10-parameter version.

Comparing the movement completion times of our tested tasks against similar tasks in the literature, our optimized controllers completed movements in approximately the same time range as other studies have reported for moderate-speed movements (Nagasaki, 1989; Uno et al., 1989). However, one limitation of a static-gain controller with step response is that movements cannot be accelerated or slowed without having to modify the controller gains.

We examined a representative subset of the movements from the Generality Test for smoothness and path efficiency. For the two optimized gain sets, trajectories tended to be smooth and efficient. For certain tasks, the optimized gain sets showed a small amount of oscillation before settling at the target position; this occurred somewhat more often in movements generated by the optimized 2-parameter gain set.

The constant steady state error values seen in all controllers (Fig. 6) indicate that the controlled hand remains static once it achieves its final position. This suggests that these controllers produce efficient movements that are terminated once the final position is attained.

This work included delay related to muscle dynamics, which was a characteristic of the muscle model used. However, we did not include neural delay; this was a reasonable assumption, as FES systems do not involve this form of delay.

Simulated annealing optimization has the disadvantage that it is a relatively slow optimization method (Rutenbar 1989). Additionally, it cannot guarantee a global optimal solution (Suman and Kumar 2006). We therefore performed several optimizations of both controllers, each with a different randomly-generated seed value, to confirm that each optimization result was consistent with the other optimization results for that controller.

Our arm model assumes a fixed scapula, which limits the range of humeral elevation. Glenohumeral stability was not considered for our system. When extending our optimized controllers to high-level SCI populations, whose rotator cuff muscles are generally not functional, adding stimulation of the rotator cuff muscles should be considered in order to ensure shoulder stability (Blana, 2008).

While PD control will always have fundamental limitations, including the inability to adapt its architecture to changing environments and arm properties, we believe that optimizing PD control can be a useful step toward developing the most accurate and efficient advanced control architectures possible. Although exact correspondence between model and human subject is unlikely, thus necessitating fine-tuning in the clinical setting, we believe that useful improvements in control can be achieved via optimization. In summary, our results demonstrate that mathematically optimizing PD control can yield substantial improvements in performance, beyond what standard tuning algorithms can achieve.

5. Conclusion

We have optimized two proportional-derivative (PD) controller gain sets on a 3-dimensional biomechanical arm model performing goal-oriented reaching movements, and have demonstrated that optimization can yield significant improvements in controller accuracy over a wide range of dynamic reaching tasks, when compared with three other PD controller gain sets that had not been optimized for this system. The optimized controllers used physiologically reasonable levels of muscular effort to perform tasks, and they maintained superior performance in the presence of variable muscular fatigue. Comparing the performance of our PD controller specified by 2 gain parameters against another requiring 10 gains, it was found that the accuracy of both controllers was very similar. It is therefore recommended that the simpler 2-parameter version of the controller be used in the future. The technique of optimizing PD control permits the use of a simple control architecture to achieve accurate and efficient upper extremity control with the goal of restoring voluntary arm movement to individuals paralyzed by high-level spinal cord injury.

Supplementary Material

1

Animation of 3-dimensional arm performing assorted reaching tasks using proportional-derivative (PD) controllers with Ziegler-Nichols-tuned vs. optimized 10-parameter gain sets. For each task, the light-colored arm is the target position. The arm's final position for each reaching task is the position in which it rests for several seconds before the next task begins.