Acceleration feedback improves balancing against reflex delay

Abstract

A model for human postural balance is considered in which the time-delayed feedback depends on position, velocity and acceleration (proportional–derivative–acceleration (PDA) feedback). It is shown that a PDA controller is equivalent to a predictive controller, in which the prediction is based on the most recent information of the state, but the control input is not involved into the prediction. A PDA controller is superior to the corresponding proportional–derivative controller in the sense that the PDA controller can stabilize systems with approximately 40 per cent larger feedback delays. The addition of a sensory dead zone to account for the finite thresholds for detection by sensory receptors results in highly intermittent, complex oscillations that are a typical feature of human postural sway.

1. Introduction

Stabilizing unstable systems is a challenging task for control engineers, and, at the same time, it is an exciting problem for computational neuroscientists [1,2]. Since the introduction of the notion ‘control of chaos’ along with the related Ott–Grebogi–Yorke method and Pyragas control of unstable motions [3], it has become clear that stabilizing unstable states is often expedient above controlling stable ones, especially from the viewpoint of required control energy. This might be the key to the success of human walking [4], and this has been crucial in many control engineering tasks [5,6]. The benchmark experimental paradigm is the stabilization of an inverted pendulum. Surprisingly, it is possible to stabilize an inverted pendulum with time-delayed feedback [7–9]. Because time delays are intrinsic components of neural control, this observation has greatly influenced the interpretation of human balancing tasks, such as stick balancing at the fingertip and postural sway during quiet standing [10–15]. Typically, these interpretations are based on a proportional–derivative (PD) controller, namely the corrective movements depend on the angular position and angular velocity [10,15–19]. However, clinical [20–22] and experimental [23–26] observations strongly suggest that balance control is benefited by mechanoreceptive (tactile, or force detectors), proprioceptive (muscle spindle) and vestibular labyrinth (otoliths and semicircular canals) inputs. In addition to sensory inputs, acceleration information can also be obtained from internal models of the biomechanics of the human inverted pendulum [27]. These observations suggest that feedback controllers must be extended to a proportional–derivative–acceleration (PDA) controller to take into account contributions owing to acceleration [23,26,28] (or force [29]). The use of PDA feedback without reflex delay for stabilizing motor plants has been discussed in the context of computational neuroscience [23,30].

It is known that a PD controller cannot stabilize an unstable equilibrium if the feedback delay is larger than a critical value [7,31]. For an inverted pendulum, the critical feedback delay can be given as

1.1

where T_p is the period of the small oscillations of the same mechanical structure hanging at its downward position [15]. Note that equation (1.1) is valid for both the cart pendulum model (∼stick balancing) and the pinned pendulum model (∼postural sway). The governing equations for the delayed PD controller take the form of a retarded functional differential equation (RFDE). Although RFDEs have infinitely many roots, only finitely many of them may have positive real parts leading to instability [32,33]. By contrast, for a PDA controller, the governing equation is a neutral functional differential equation (NFDE), because the delay is present in the argument of the highest derivative, namely the acceleration. An NFDE can have infinitely many roots with positive real part. Thus, apart from the usual engineering objection of using the ‘noisy’ acceleration signals in feedback loops, the control design of such systems requires additional care. The complexities of NFDEs have fuelled an enormous mathematical literature [7,32–34]; however, there have been few practical applications (for notable exceptions, see recent studies [35–38]). Here, we show that the stability criteria for the NFDE that arises in the setting of an inverted pendulum stabilized by a corrective torque can be readily determined.

While balancing can be described by linear (or linearized) governing equations, if the postural sway is limited to small angular displacements, there exists a strong small-scale nonlinearity related to the sensory threshold [39–41]. A consequence of the presence of such sensory dead zones is that closed-loop feedback control occurs only when inputs exceed sensory thresholds. While this nonlinearity has no effect on large-scale stabilization in the linear system, it may lead to small amplitude chaotic oscillations (also called micro-chaos) as explained and proved for robotic systems in earlier studies [42–44].

The paper is outlined as follows. First, the concepts of feed-forward, delayed state feedback and predictive controllers are discussed. It is argued that prediction based on the delayed state is equivalent to the direct feedback of the delayed state. In §3, the mechanical model of postural sway is presented involving the reaction time and the sensory dead zones. A delayed PDA control law is applied; thus, the governing equation is an NFDE. In §4, the criteria for the stabilizability of the system are derived. Section 5 presents a numerical study for human postural sway. The results are discussed and concluded in §6.

2. Background

Delayed feedback controllers can be classified as either delayed state feedback or predictive controllers. Predictive controllers, which predict the actual state of the system based on its delayed states, are practically equivalent to the direct feedback of the delayed states. It is a rule of thumb that the more that recent information is used in the controller, the better the performance of the control system. For this reason, we will concentrate on control laws, for which the state variable from time instant t−τ is fed back with the feedback delay τ. Including terms higher than the second derivative such as the jerk results in an unstable system, because the governing equation is a functional differential equation of advanced type [45]. For this reason, we will concentrate on control laws, where only the position, the velocity and the acceleration are fed back.

In order to appreciate the relationship between a PDA feedback controller and predictive control, first the concepts of feed-forward, feedback and predictive control are reviewed briefly. Then, these concepts are applied to the balancing task.

2.1. Feed-forward, feedback and predictive controllers

Consider a linear system

2.1

where is the state, is the input, A is the state matrix, B is the input matrix and the notation denotes (d/dt)x(t). The task is to find an input u(t) such that the system is moving according to the desired path x_d(t). We assume that the state x is completely observable without any noise and there is no need to apply state estimators such as the Kalman filter. The control law can be written in the form

2.2

where u_ff(t) is the feed-forward term and u_fb(t) is the feedback term [46]. The feed-forward term can be given as

2.3

If x_d(t), , A and B are known perfectly, then a pure feed-forward controller without any feedback term (i.e. u(t) = u_ff(t)) drives the system along the desired path, provided that there are no uncertainties in the system parameters, the feed-forward calculation is totally accurate, there are no perturbations or noise affecting the system, and the initial conditions fit to the desired motion perfectly. The pure feed-forward controller is an open-loop controller, because no information about the state is fed back.

In the presence of parameter uncertainties or perturbations, the feedback term u_fb(t) should also be applied in order to compensate the deviation from the desired path using the error

2.4

Equations (2.1) and (2.3) imply the error dynamics, which are governed by

2.5

This equation is also called the variational system of (2.1) around the desired motion x_d(t). Application of both feed-forward and feedback control terms (i.e. u(t) = u_ff(t)+u_fb(t)) drives the system along the desired position, eliminating the error e(t) = x(t)−x_d(t), provided that the error dynamics governed by (2.5) are stable. Therefore, the crucial point is to find a feedback controller u_fb, which stabilizes (2.5). The difficulties arise because the feedback loops inherently involve time delays which can destabilize the system.

An example for the delayed state feedback is

2.6

where τ is the feedback delay and matrix D contains the control gains. In this case, the stability of the system is determined by the RFDE

2.7

Predictive controllers suggest that, rather than feeding back the delayed state, one should predict the actual state based on the known time history (figure 1). If the state is predicted based on its own delayed values only, and the feedback term u_fb is not involved in the prediction, then the predicted error can typically be written as

2.8

where P(τ) is a matrix describing the prediction over the delay period. Feedback of the predicted error according to the linear control law gives

2.9

which is equivalent to equation (2.7) with . This demonstrates that prediction based on the state is equivalent only to the direct feedback of the delayed state.

Figure 1.

Scheme of known and unknown states owing to feedback delay.

If the feedback term u_fb is also involved into the prediction through an efferent copy of corollary discharges of the motor command, then the feedback term can be written in the general form

2.10

where f is a measurable function, and τ_x and τ_u are the delays in obtaining the state and the feedback information, respectively. Here, the predicted state is determined as a weighted integral over the past using the state x over the interval [0,t−τ_x], the feedback term u_fb over [0,t−τ_u] and the desired state x_d, which is known over [0,t]. In several models, it is assumed that τ_u = 0, which requires that u_fb(t) is readily available. When τ_u = 0, an appropriate choice of the function f may lead to a finite spectrum assignment, provided that all the parameters are exactly known, the numerical calculation in (2.10) is performed with perfect accuracy and there are no perturbations [47,48]. In fact, (2.5) and (2.10) form an NFDE, which may face instabilities owing to the slightest uncertainties in the system parameters due to an unstable pole-zero cancellation [49,50]. Therefore, in this analysis, we chose the prediction based on the delayed states only, which is equivalent to a direct delayed state feedback.

2.2. Application to the balancing problem

We can apply the earlier mentioned concepts to the case of balancing an inverted pendulum described by the linearized equation

2.11

where φ is the small angular position measured from the φ = 0 vertical upright position (see §3), is the angular acceleration, α > 0 is the system parameter, and Q(t) = Q_ff(t) + Q_fb is the control torque with Q_ff(t) and Q_fb being the feed-forward and the feedback terms, respectively. Here, the state is described by the vector , where is the angular velocity and T denotes the transpose.

If the desired angular motion is given as φ_d(t), then the feed-forward control torque should be given as . If the system parameter α is known precisely, no noise is affecting the system and the initial conditions fit perfectly to φ_d(t), then the control torque Q(t) = Q_ff(t) drives the system along the desired path. In the presence of parameter uncertainties and noise, however, the feedback term Q_fb(t) is also required to reduce the error e(t) = φ(t)−φ_d(t).

In balancing, the desired position is the vertical upright position, i.e. ; consequently, . Hence, the error is equal to the state itself (e(t) = φ(t)−φ_d(t) = φ(t)), and the error dynamics are governed by

2.12

Owing to the feedback delay, the state variables describing the pendulum's motion (e.g. angular position φ, angular velocity or angular acceleration ) are available only in the interval [0,t−τ], and no information can be obtained from the interval (t−τ, t] (figure 1). The task is to find a stabilizing feedback control torque Q_fb using the available delayed information only.

Linear feedback of a simple linear prediction of the angular position and a constant prediction of the angular velocity implies the equation

2.13

where and are the proportional (position) and the derivative (velocity) gains for the predicted state. This equation is equivalent to the direct feedback of the delayed position and the delayed velocity (PD feedback) with proportional gain and with derivative gain .

A second-order prediction of the angular position can be given using the delayed angular acceleration as , a linear prediction of the angular velocity can be given as and a constant prediction of the angular acceleration is . Linear feedback of these terms results in

2.14

where is the acceleration gain for the predicted state. This equation is equivalent to the direct feedback of the delayed position, velocity and acceleration (PDA feedback) with proportional gain , derivative gain and acceleration gain . In this sense, delayed PDA feedback can be interpreted as a feedback of a special prediction of the current position, velocity and acceleration based on their delayed values. Note that, independently of this interpretation, the control gains (k_p, k_d, k_a) or, alternatively, ( should be tuned in order to achieve stability.

3. Model

The mechanical model of balancing in the sagittal plane via torques applied at the ankle joint is shown in figure 2. Following Winter et al. [51] and Loram & Lakie [52], the human body is modelled as a rod of mass m pivoted on joint A. The distance between the centre of gravity C and the suspension point A is denoted by ℓ_AC. J_C is the moment of inertia with respect to the normal line via the centre of gravity. The passive but insufficient resistance of the ankle joint against falling is modelled by a torsional spring of stiffness k_t and a torsional dashpot of damping b_t. These elements are attributed to the foot, Achilles tendon and aponeurosis, and they cannot be neurally regulated during quiet standing. As shown by Loram & Lakie [52], the stiffness increases slightly with ankle torque, which verifies the linear spring model. They also showed that the intrinsic mechanical stiffness of the ankle is insufficient for stability during quiet standing, and additional modulation of parallel connected calf muscle fibres is required to maintain balance. This torque acting on the body is denoted by Q in figure 2. The angle of the human body with respect to the vertical is denoted by φ. The equation of motion can be written in the form

3.1

where J_A = J_C + mℓ_AC² is the moment of inertia of the body with respect to the normal line via the pivot point A, and g = 9.81 m s⁻² is the gravitational acceleration.

Figure 2.

Mechanical model of postural balancing.

The control torque Q(t) is assumed to be a linear combination of the angular position φ, the angular velocity and the angular acceleration . These quantities are obtained from mechanoreceptive (tactile, or force detectors) and proprioceptive (muscle spindle) sensors, from vestibular organs (otoliths and semicircular canals) and from visual inputs. The presence of a sensory dead zone is accounted for by assuming that the actuating forces occur only if the input signals exceed some threshold values [12,18,40,53]. Furthermore, the overall reaction time is modelled as a feedback delay. Assuming different thresholds for the different sensation inputs, the control torque can be given as

3.2

with

3.3

3.4

and

3.5

where K_p is the proportional gain, K_d is the derivative gain, K_a is the acceleration gain, τ is the feedback delay, and φ_s, and are the sensory threshold values for the angular position, the angular velocity and the angular acceleration, respectively. Note that the delay τ appears also for threshold conditions.

An important element of the current analysis is that the control force is activated only for motions exceeding some thresholds [12,40]. For small motions, when the state variables are within the sensory dead zone, there is no control action, and the system behaviour is determined by the unstable open-loop system. A reasonable claim is that equation (3.1) with (3.2)–(3.5) cannot have a domain of attraction around if the linear system

3.6

is not stable. This is proved in Haller & Stepan [42] and Enikov & Stepan [43] for digital balancing where the time delay is not constant but periodically varying owing to digital sampling. Therefore, the stability analysis of (3.6) plays a crucial role in the problem. If (3.6) is stable, then the nonlinear system (3.1) with (3.2)–(3.5) has an attractor (a limit cycle or a chaotic attractor) around . If (3.6) is unstable, then (3.1) with (3.2)–(3.5) typically does not have any attractor and the balancing process ends up with falling. Note that the system without the control torque Q(t) is unstable, because the intrinsic mechanical stiffness of the ankle alone cannot maintain stability, i.e. k_t−mgℓ_AC < 0 [52].

This controller is intermittent in the sense that the control force is switched on and off depending on the size of the sensory inputs. Although this kind of switching does not affect stability, there are other types of switching conditions that may strongly contribute to the stability of the system. For instance, Asai et al. [13] applied an intermittent controller, which uses the stable manifold of the saddle of the open-loop system (see model 3 in their paper). If the system is close to the stable manifold, then the control is switched off, and it is switched on only when it gets close to the unstable manifold. The corresponding switching condition is defined in the phase plane: if (with ) then the controller is switched on; otherwise, the controller is switched off. They showed that the resultant intermittent system is more stable than the linear system. In our approach, the intermittency appears as a result of the modelling of the sensory dead zone, which is also modelled by Asai et al. [13] in their model 4.

By rescaling the time as and dropping the tilde immediately, (3.6) can be transformed into the form

3.7

where

3.8

and

3.9

Equation (3.7) is an NFDE, because the second derivative of the state variable () appears both with actual and with delayed arguments. Consequently, infinitely many unstable characteristic exponents may arise for certain parameter combinations. In what follows, we will determine the critical maximum value of the delay τ for which the system can still be stabilized.

4. Criteria for stabilizability

The criteria of stabilization are presented for four cases: (i) the undamped case with PD control (b = 0, k_a = 0); (ii) the damped case with PD control (b ≠ 0, k_a = 0); (iii) the undamped case with PDA control (b = 0, k_a ≠ 0); and (iv) the damped case with PDA control (b ≠ 0, k_a ≠ 0).

4.1. Proportional–derivative control for the undamped case (b = 0, k_a = 0)

If k_a = 0 then (3.7) is an RFDE. For this system, the criteria for the stabilizability are known in the literature [7,31,54]: if the dimensionless system parameter a is larger than a critical value given by

4.1

then the system is unstable for any k_p and k_d. Note that this result can also be obtained as a special case k_a = 0 of the analysis of the PDA controller in §4.3 and appendices A.1–A.3. From here, simple calculation gives that the critical feedback delay for the balancing task governed by (3.6) with b_t = 0 and k_a = 0 is

4.2

Note that the period of the small oscillations of the pendulum hanging at its downward position is ; thus (4.2) is equivalent to (1.1).

4.2. Proportional–derivative control for the damped case (b ≠ 0, k_a = 0)

If there is a damping in the system, then the critical value for the dimensionless system parameter a is

4.3

which implies the critical feedback delay

4.4

For small values of b_t, the critical delay is slightly larger than it is for the undamped case.

4.3. Proportional–derivative–acceleration control for the undamped case (b = 0, k_a ≠ 0)

If k_a ≠ 0, then (3.7) is an NFDE. It is known that if |k_a| > 1, then (3.7) has infinitely many characteristic roots with positive real parts (see lemma 3.9 on p. 63 in Stepan [7]); therefore, in the current analysis, we concentrate only on the case |k_a| < 1. Stability analysis for (3.7) with |k_a| < 1 can be performed according to the D-subdivision method combined with the analysis of the exponent-crossing direction along the D-curves [7]. Briefly, the D-curves separate the plane of some selected system or control parameters (in this case, the plane (k_p, k_d)) into domains where the numbers of unstable characteristic exponents are constant. The stability boundaries are the D-curves bounding the domains with zero unstable characteristic exponent.

The D-curves for (3.7) can be given as

4.5

and

4.6

where ω is the frequency parameter, which is equal to the imaginary part of the characteristic exponent λ, i.e. ω = Im(λ) (for details, see appendix A.1). The D-curve k_p = a given by (4.5) is associated with a real critical characteristic exponent λ = 0, whereas the D-curve given by (4.6) is associated with a complex conjugate pair of characteristic exponents of the form λ = ±iω. In fact, the parametric curve (4.6) defines a spiral in the plane (k_p, k_d), as shown in figure 3. The number of unstable characteristic roots in the individual domains separated by the D-curves can be determined by the analysis of the exponent-crossing direction (for details, see appendix A.2). The corresponding stability chart with the number of unstable exponents for a = 2 and k_a = 0.5 is presented in figure 3. Some specific values of the frequency parameter ω along the stability boundaries are also presented. The stable domain (with zero unstable characteristic root) is indicated by grey shading in figure 3b (zoomed).

Figure 3.

Stability chart with the number of unstable characteristic exponents for (3.7) with a = 2, b = 0 and k_a = 0.5 (PDA controller).

It is shown in appendix A.3 that, as the dimensionless system parameter a gets larger and larger, the domain of stability gets smaller and smaller. If a is larger than a critical value given by

4.7

then the system is unstable for any k_p, k_d and k_a. Consequently, the critical feedback delay for the balancing task governed by (3.6) with b_t = 0 can be given as

4.8

that is, (see (4.2)). This means that the critical delay for the PDA controller is larger by approximately 40 per cent than that of the PD controller. Note that this result corresponds to the one obtained by Sieber & Krauskopf [28].

4.4. Proportional–derivative–acceleration control for the damped case (b ≠ 0, k_a ≠ 0)

The stability criteria for the damped case can be derived in a similar way; the only difference is that the term bλ shows up in the characteristic function. The D-curves in this case form as

4.9

and

4.10

The critical system parameter, at which the stable domain in the stability chart disappears, can be obtained as a_{crit,PDA,damped} = 2 + 2k_a + 2b, which, considering that |k_a| < 1, implies

4.11

Thus, the damping term also contributes to the stabilizability of the system.

Using (3.8), the critical feedback delay for the balancing task governed by (3.6) can be given as

4.12

For reasonably small damping values, this critical delay is larger by approximately 40 per cent than that of the corresponding PD controller.

5. Numerical examples

In this section, the stability diagrams are determined for some specific parameter values from the corresponding literature. First, the parameters from Asai et al. [13] are used to determine the stability charts for the PD and the PDA controller. Then, two parameter sets are taken from Loram & Lakie [52] and a numerical simulation is presented to show the effect of the sensory dead zone.

5.1. Case study 1

The mechanical and physiological parameters for the first case study were taken from Asai et al. [13] (table 1). The corresponding stability diagrams can be seen in figure 4. The stability regions (shaded) were determined using (4.9) and (4.10). It can be seen that the stability domain for the PDA feedback is larger than that of the PD controller. Larger stable domains provide greater robustness against parameter perturbations. A comparison with the stability charts in Asai et al. [13] shows that their intermittent controller (model 3 in their paper) is able to stabilize the system for control gains for which the linear PD and PDA controllers are unstable. The stability boundaries for the PD controller obtained by the approximation of the delayed terms by their first-order Taylor series expansion is also presented for reference (for details, see Asai et al. [13]). As can be seen, this approximation is appropriate only for small control gains.

Table 1.

Mechanical and physiological parameters taken from Asai et al. [13].

m = 60 kg	bt = 4.0 Nms rad−1
ℓAC = 1 m	τ = 0.2 s
JA = 60 kg m2	a = 0.0784
kt = 471 Nm rad−1	b = 0.013

Figure 4.

Stability charts for PD and PDA controllers with the parameters listed in table 1. Stable domains are denoted by grey shading. Dashed lines denote the stability boundaries for the PD controller obtained by the approximation of the delayed terms by their first-order Taylor series expansion. The acceleration feedback gain for the PDA controller is K_a = 54 (Nms² rad⁻¹) such that its dimensionless value is k_a = 0.9.

5.2. Case study 2

For the second case study, the parameters were taken from Loram & Lakie [52], in which the parameters of 15 subjects were measured. Two sets of parameters were selected: (i) the mean value of the 15 subjects and (ii) subject JR, who had the smallest passive resistance against falling. The corresponding parameters are listed in table 2. The reflex delay was chosen to be τ = 200 ms as a mean value from Loram & Lakie [52]. The stability charts for the PD and the PDA controllers are presented in figure 5. Because the combined relative stiffness for subject JR is the smallest, the stable domain is smaller than that determined from the mean values and it is also shifted towards larger K_p values.

Table 2.

Mechanical and physiological parameters for the numerical study taken from Loram & Lakie [52].

dataset	mean value	subject JR
m (kg)	75.5	70.9
ℓAC (m)	0.92	0.87
JA (kg m2)	63.9	53.66
kt (Nm rad −1)	595.5	217.7
bt (Nms rad −1)	4.011	4.011
a	0.0535	0.2877
b	0.0126	0.0149

Figure 5.

Stability charts for PD and PDA controllers with the parameters listed in table 2. Stable domains are denoted by grey shading. The acceleration feedback gain is K_a = 57.51 (Nms² rad⁻¹) for the mean value and K_a = 48.29 (Nms² rad⁻¹) for subject JR such that the corresponding dimensionless values are K_a = 0.9 for both cases. (a) Mean values and (b) subject JR.

Time domain simulations for the nonlinear system (3.1) with (3.2)–(3.5) were performed using the zeroth-order semi-discretization method described by Insperger & Stepan [54]. The parameters of subject JR were used for the simulation. The control gains are given in table 3. The discretization step was 1 ms. The sensory threshold values were set to φ_s = 0.004 (rad), (rad s⁻¹) and (rad s⁻²) [13]. The initial conditions for the simulation were φ(0) = 0.001 rad, and .

Table 3.

Control gains for the simulations and the corresponding dimensionless values.

Kp = 804.9 Nm rad−1	Kp = 0.6
Kd = 268.3 Nms rad−1	Kd = 1
Ka = 48.29 Nms2 rad−1	Ka = 0.9

The results of the simulations are shown in figure 6. The angular acceleration and the control torque Q show strong discontinuities denoted by arrows in figure 6. This feature originates from two sources. First, the sensory dead zone gives an intermittent nature to the controller, which typically results in an almost periodic motion (in a limit cycle) [18,39–41]. Second, it is a special feature of NFDEs that the initial discontinuities do not decay in time (as opposed to the RFDEs) [32,33]. These two effects result in a highly intermittent and chaotic-like signal. Solutions of this type have been referred to as bounded, time-dependent states [58]. Similar behaviour is typical in robotic balancing, where the digital effects, i.e. the sampling, the processing delay and the round-off error, result in a small amplitude chaotic motion around the equilibrium, which is called micro-chaos [42–44]. In these applications, the round-off error presents a dead zone around equilibrium similar to the sensory dead zone described by (3.3)–(3.5).

Figure 6.

Time history and the control torque for balancing via the PDA controller involving feedback delay and sensory thresholds. The sensory dead zones are denoted by grey shading. Arrows denote discontinuities in the acceleration and the control torque signal.

Similar discontinuities might also arise in models using intermittent control laws, such as an intermittent predictive controller [55,56], an act-and-wait controller [54,57] or a stochastic drift-and-act controller [2,12,58]. Indeed, the experimental demonstration of the intermittent nature of human balance control tasks [11,13,59] raises the possibility that such control strategies may be of widespread use by the nervous system.

6. Conclusion

Here, we have shown that acceleration feedback benefits the stabilization of the upright position of an inverted pendulum. In the absence of time delay, the acceleration feedback acts as an active inertia to reduce the spring constant [60]. However, in the presence of time delays, the applied PDA controller is equivalent to a predictive controller where the actual state is predicted based on the delayed position, velocity and acceleration. If the jerk, or higher order derivatives, is also involved in the prediction, then the linear system becomes unstable because the resultant functional differential equation becomes of advanced type. Thus, direct feedback of the delayed position, velocity and acceleration with different feedback gains gives an optimal variety for controllers using predictions based on the observed state in the absence of noise. In order to achieve a stable process, the three control gains should be adjusted, possibly by a learning process, and no a priori knowledge is necessary about the system parameters. This approach is in contrast to optimal control using model-based predictive controllers [10,61,62]. However, it has been shown that human postural responses to support–surface translation predicted by equations of the form we have discussed here resemble very closely those predicted from optimal control [26].

If there is noise in the system or the state is not completely observable, then the application of state estimators such as the Kalman filter is a reasonable solution [10]. In this case, the state estimation should be extended to the acceleration as well in addition to the position and the velocity estimation. However, once the position, velocity and acceleration are estimated in an optimal way, the PDA controller provides better stability properties than the corresponding PD controller, which is a straight consequence of the current study.

The criteria τ < τ_crit (or alternatively, a < a_crit) guarantee the stability of the vertical fixed point (φ = 0) of the inverted pendulum. However, experimental observations suggest that the fluctuations in φ for human postural sway and stick balancing at the fingertip do not resemble those expected for a stable fixed point subjected to noisy perturbations. In particular, a rhythmic, or possibly chaotic, component has been emphasized (see [58]). One possible interpretation is that these complex dynamics reflect the fact that sensory thresholds are finite and hence sensory dead zones will be present [18,40]. We have shown that the addition of such thresholds to a PDA controller is sufficient to produce very complex and intermittent types of oscillatory behaviours. However, we suggest that the stability of the upright position, at least in the mathematical sense, is determined by the stability of the dynamical system when the sensory dead zones have zero size. In other words, the addition of a sensory dead zone cannot itself stabilize an unstable upright position. On the other hand, it is possible that a sensory dead zone can be destabilizing. This is because the amplitude of the complex oscillations is proportional to the magnitude of the sensory dead zone. Large-amplitude fluctuations may exceed the basin of attraction for the upright position and cause a fall [58]. The beneficial effects of a sensory dead zone become manifest when, for example, the interactions between delay and noise transiently reduce the amplitudes of trajectories below sensory thresholds [2]. In such cases, balance is achieved without active neural feedback and hence at minimal metabolic expense.

Our observations do not eliminate the role of intermittent controllers in balance control [13,58]. Indeed, the existence of ankle–hip–step strategies to maintain balance in response to increasingly large perturbations [63] strongly points to a nested, or ‘safety net’, control network topology. In other words, the number of feedback loops that participate in control increases as the vertical displacement angle increases [64]. An important distinguishing point is that the thresholds in these strategies are not related to sensory dead zones, but rather are likely to be set in a manner to influence stability.

Although the potential advantages of acceleration feedback have long been recognized [23,29,60], only with the availability of inexpensive triaxial accelerometers have their beneficial effects on balance control been investigated experimentally [65,66]. The fact that the delayed PDA controller is superior to the delayed PD controller with respect to stabilizability suggests that other fields of neuroscience in which PD controllers have been used previoulsy could also benefit by involving acceleration feedback in the control law.

Appendix

A.1. Determination of the D-curves

The characteristic function for (3.7) reads

A1

According to the D-subdivision method, substitution of λ = γ± iω, ω ≥ 0 into D(λ) = 0 and decomposition into real and imaginary parts give

A2

and

A3

Substitution of γ = 0 into (A 2) and (A 3) gives the D-curves given in (4.5) and (4.6).

A.2. Determination of the number of unstable characteristic exponents

As shown in figure 3, the D-curve (4.6) defines a spiral in the plane (k_p, k_d), while (4.5) gives a vertical line. Simple analytic calculation shows that the distance between the point (k_p, k_d) = (a,a) and the intersection points along the line k_p = a is increasing with each cycle of the spiral, provided that |k_a| < 1. In order to see this, let ω_j, denote the parameters for which the curve (4.6) intersects the line k_p = a. Odd values of j result in positive k_d, while even values of j give intersections with negative k_d. For k_d > 0, the distances of the intersection points tend to

A4

where with . For , a similar calculation gives

A5

Consequently, the curves (4.5) and (4.6) cut the parameter plane (k_p, k_d) into infinitely many domains that are all attached to the line k_p = a.

In general, the number of unstable characteristic roots for the individual domains can be determined by the analysis of the exponent-crossing direction (also called root-crossing direction or root tendency) along the D-curves, which is the sign of the partial derivative of the real part of the characteristic exponent with respect to one of the system parameters. If the number of unstable exponents is known for at least one point in one domain, then it can be determined for all the other domains by considering the exponent-crossing direction along the D-curves.

Here, the exponent-crossing direction along the D-curve k_p = a is determined by taking the partial derivatives of (A 2) and (A 3) with respect to k_p and by substituting γ = 0, ω = 0 and k_p = a. This derivation gives

A6

which means that γ ′_kp is positive for k_d < a and negative for k_d > a. If the line k_p = a is crossed from left to right with k_d > a, then a real characteristic exponent becomes stable. If k_d < a, then a real exponent becomes unstable as the line k_p = a is crossed from left to right.

If k_p = 0, k_d = 0 with and a > 0, then the number of unstable characteristic exponents is 1 (see fig. 3.10 on p. 64 in Stepan [7]). The number of unstable exponents can be given for all the domains in the parameter plane (k_p, k_d) by considering the exponent-crossing directions along the D-curve k_p = a.

Note that the number of unstable exponents can also be determined using Stepan's formula (see theorems 2.15 and 2.16 and the proof of theorem 2.20 in Stepan [7]).

A.3. Determination of the critical system parameter

In order to determine the critical system parameter a_crit,PDA, which limits the stabilizability of (3.7), the D-curves (4.5) and (4.6) should be analysed for different values of a and k_a. Series of stability diagrams with different dimensionless system parameters a and acceleration control gain k_a are presented in figures 7 and 8 for different control parameter regions. Figure 7 shows that the intersection points along the line k_p = a are more and more dense as k_a gets closer and closer to 1.

Figure 7.

D-curves and the number of unstable characteristic exponents for (3.7) with different dimensionless system parameters a and acceleration control gains k_a (PDA controller) and b = 0.

Figure 8.

Stability charts zoomed on the domain of stability for (3.7) with different dimensionless system parameters a and acceleration control gains k_a and b = 0. Numbers indicate the number of unstable characteristic exponents.

Note that if k_a > 1, then the system has infinitely many unstable characteristic exponents. Figure 8 shows diagrams zoomed on the domain of stability (with zero unstable exponent). It can be seen that, as the system parameter a gets larger and larger, the domain of stability gets smaller and smaller and disappears when the tangent of the parametric curve (4.6) at ω = 0 becomes vertical. A long but straightforward algebraic derivation gives

A7

The tangent is vertical if 6−6k_a−3a = 0, which gives the critical value a_crit,PDA = 2 + 2k_a. Because the only limitation to k_a was that |k_a| < 1, the critical value for the system parameter a in the case of a PDA controller is

A8

If and , then the domain of stability shrinks to the point (k_p, k_d) = (4,4). In the limit case a = 4, k_a = 1 and (k_p, k_d) = (4,4), there are infinitely many characteristic exponents with zero real part.

An alternative way to determine the critical value for a is the Taylor series expansion of k_p(ω) in (4.6),

A9

If the coefficient of ω² is positive, then the proportional gain k_p is increasing with increasing ω, which gives the same critical value a_crit,PDA = 2 + 2k_a.