Real-time simultaneous and proportional myoelectric control using intramuscular EMG

Abstract

Objective

Myoelectric prostheses use electromyographic (EMG) signals to control movement of prosthetic joints. Clinically available myoelectric control strategies do not allow simultaneous movement of multiple degrees of freedom (DOFs); however, the use of implantable devices that record intramuscular EMG signals could overcome this constraint. The objective of this study was to evaluate the real-time simultaneous control of three DOFs (wrist rotation, wrist flexion/extension, and hand open/close) using intramuscular EMG.

Approach

We evaluated task performance of five able-bodied subjects in a virtual environment using two control strategies with fine-wire EMG: (i) parallel dual-site differential control, which enabled simultaneous control of three DOFs and (ii) pattern recognition control, which required sequential control of DOFs.

Main Results

Over the course of the experiment, subjects using parallel dual-site control demonstrated increased use of simultaneous control and improved performance in a Fitts' Law test. By the end of the experiment, performance using parallel dual-site control was significantly better (up to a 25% increase in throughput) than when using sequential pattern recognition control for tasks requiring multiple DOFs. The learning trends with parallel dual-site control suggested that further improvements in performance metrics were possible. Subjects occasionally experienced difficulty in performing isolated single-DOF movements with parallel dual-site control but were able to accomplish related Fitts' Law tasks with high levels of path efficiency.

Significance

These results suggest that intramuscular EMG, used in a parallel dual-site configuration, can provide simultaneous control of a multi-DOF prosthetic wrist and hand and may outperform current methods that enforce sequential control.

1. Introduction

Upper extremity amputation proximal to the wrist is a source of disability for approximately 41,000 people in the United States (Ziegler-Graham et al., 2008), including many veterans injured in conflicts overseas (Fischer, 2013). Approximately 40% of such amputations are at the transradial level (Dillingham et al., 2002). Myoelectric prostheses, controlled using surface electromyographic (EMG) signals from residual muscles, have the potential to enable intuitive control for transradial amputees because EMG signals from physiologically appropriate muscles are used to control the prosthetic wrist and hand. However, up to 40% of transradial amputees reject their myoelectric prostheses (Biddiss and Chau, 2007) and desire improved control of prosthetic joints (Biddiss et al., 2007). A notable limitation of clinically available myoelectric control methods is that patients must control each degree of freedom (DOF) sequentially, i.e., one at a time. Neither conventional dual-site differential control (Williams, 2004) nor newly available pattern recognition control (Coapt, 2013, Kuiken et al., 2009) allow simultaneous control of DOFs. This prevents prosthesis users from experiencing the coordinated joint control possible in an intact limb. Although advanced arm systems (Harris et al., 2011, Kyberd et al., 2011, Resnik, 2011), including multi-DOF wrists, offer the mechanical means to restore such movements, there is still a significant need for systems that enable simultaneous control of such devices.

A variety of approaches to providing simultaneous control have been investigated using surface EMG. Such approaches have included pattern recognition (Young et al., 2012, Wurth and Hargrove, 2013), use of neural networks to predict joint kinematics (Muceli and Farina, 2012, Jiang et al., 2012) or kinetics (Nielsen et al., 2011), and analysis of underlying muscle synergies (Jiang et al., 2009, Choi and Kim, 2010). These approaches are promising, but most studies have been limited to controlling the wrist without the hand, or have enforced equal velocities on all active DOFs, and thus do not provide independent proportional control of each DOF. Many previous studies have also been confined to offline evaluation and thus have not demonstrated simultaneous control in real-time. However, real-time evaluation is important because simply providing a simultaneous control system may not result in improved real-time performance; poor controllability, non-intuitiveness, or reluctance to use the simultaneous control could potentially lead to poorer performance compared to sequential control in multi-DOF tasks.

Recording the EMG intramuscularly, instead of from the skin surface, has several advantages over surface EMG for providing simultaneous control. EMG signals from deep muscles are significantly attenuated at the skin surface, whereas intramuscular recording can acquire signals from the deepest muscles. Furthermore, intramuscular recording have limited crosstalk (Basmajian and De Luca, 1985). Intramuscular recording thus allows EMG signals to be obtained from any muscle in the residuum with minimal noise from the surrounding musculature. In transradial amputees, intramuscular EMG may thus allow intuitive, simultaneous control of a multi-DOF wrist-hand prosthesis. Using an approach termed “parallel dual-site” control (similarly described by Stein et al (1980) and Tucker and Peteleski (1977)), the difference in intramuscular EMG amplitudes from an antagonist-agonist muscle pair (i.e., from dual sites) controls a physiologically appropriate DOF in the prosthesis. Because cross talk is minimized, EMG from multiple muscle pairs can be used in parallel to simultaneously control multiple DOFs.

In the past, control methods using intramuscular EMG were largely overshadowed by those using surface EMG because of a lack of clinically available implantable EMG recording devices. However, a variety of implants currently being developed (Bercich et al., 2012, Weir et al., 2009, Young, 2009, McDonnall et al., 2012) may make intramuscular EMG clinically feasible as a signal source; one device was recently shown to be successful in a human subject (Merrill et al., 2011). Furthermore, chronic intramuscular implants could address several of the issues that impair the performance of surface EMG– based systems. A chronic intramuscular implant may provide a more stable, chronic signal source that is less affected by electrode shift or changes in skin impedance caused, for example, by perspiration. The recent advances in implantable recording devices and their potential advantages suggest that intramuscular EMG–based myoelectric control systems could have considerable clinical impact.

Using parallel dual-site control with intramuscular EMG has been frequently proposed (Weir et al., 2009, Merrill et al., 2011, Tucker and Peteleski, 1977, Stein et al., 1980) but has not been evaluated in depth. Most recent studies on intramuscular EMG control have instead focused on the machine-learning algorithms that are also being pursued for surface EMG (Kamavuako et al., 2012, Kamavuako et al., 2013b, Smith and Hargrove, 2013). However, the few studies of parallel dual-site control have been promising. Studies in able-bodied subjects have demonstrated the potential of this approach with intramuscular EMG to provide simultaneous finger control (Birdwell, 2012). In addition, parallel dual-site control using surface EMG was successfully implemented in above-elbow amputees who had undergone targeted reinnervation (Kuiken et al., 2009)—a surgical procedure that creates new myoelectric control sites. After surgery, these individuals had greater spatial separation between myoelectric control sites (and thus less crosstalk) than is typical. Parallel dual-site control was also implemented with experimental implantable electrodes in two transradial amputees, but these studies did not quantitatively evaluate prosthesis control or investigate simultaneous control (Stein et al., 1980, Tucker and Peteleski, 1977).

The objective of this study was to evaluate the potential for a parallel dual-site system, using intramuscular EMG, to provide simultaneous control of multiple DOFs. Five able-bodied subjects used intramuscular EMG to control a wrist-hand system with three DOFs (wrist rotation, wrist flexion/extension, and hand open/close) in a virtual environment. For comparison, subject performance was evaluated relative to using a commercially available sequential control method (pattern recognition, Coapt (2013)) implemented with intramuscular EMG. We hypothesized that the simultaneous control provided by parallel dual-site control would result in better performance in real-time tasks compared to the sequential control afforded by pattern recognition.

2. Methods

The following experiment was approved by the Northwestern University Institutional Review Board. Subjects participated after informed consent. Real-time controllability of a wrist-hand system, using parallel-dual-site control or pattern recognition control was evaluated in five non-amputee subjects. The experiment comprised two separate sessions, spaced a minimum of one week apart. Subjects used one type of control system during each session; the order in which the control systems were used was randomized across subjects. Each subject had experience using dual-site differential control and pattern recognition control with surface EMG prior to the experiment. Real-time performance using the two control methods was evaluated through two tasks: (i) the Motion Test (Kuiken et al., 2009), which evaluated how well subjects could perform an isolated, intended 1-DOF movement (e.g. wrist flexion) and (ii) a three dimensional Fitts' Law style task, which provided a real-time evaluation of prosthesis control using multiple DOFs.

2.1. Signal Acquisition and Processing

Intramuscular EMG signals were recorded using percutaneous fine wire electrodes (Natus Neurology Inc. and Motion Lab Systems Inc.) that were inserted using 25 ga hypodermic needles. Bipolar electrodes were inserted into six forearm muscles: pronator teres, supinator, flexor carpi radialis, extensor carpi radialis longus, flexor digitorum profundus, and extensor digitorum communis. Insertion sites were guided by palpation, and verified by electrical stimulation and EMG activity during test contractions. Signals were collected using a Motion Lab Systems MA300 EMG system, where signals were amplified 350×, band-pass filtered between 10-2000 Hz, and sampled at 5 kHz by a National Instruments data acquisition system (NI-USB 6218). Signals were also digitally high-pass filtered using a 3^rd order Butterworth filter with a cutoff frequency of 20 Hz. After the fine wire electrodes were inserted, subjects were restrained in a custom brace to ensure isometric contractions.

2.2. Control Strategies

For both control systems, subjects practiced the Fitts' Law task before beginning the experimental trials. Subjects were allowed to retrain the pattern recognition system or reset gains and thresholds for the parallel dual-site control system during the practice session until they were satisfied with the control system. After Motion Test and Fitts' Law blocks commenced, no modifications were made to the control systems.

2.2.1. Parallel Dual-Site Control

Each of the three DOFs was controlled using the intramuscular EMG from a physiologically-appropriate antagonistic muscle pair (Fig. 1). The mean absolute values (MAV) of the EMG signals were calculated from 250 ms sliding windows, with a frame increment of 50 ms. The MAVs of the antagonistic muscle pairs were amplified and thresholds were applied. The difference in signal amplitude was then determined and used to proportionally control the output velocity at the corresponding DOF via a linear mapping. More explicitly:

Fig. 1. Configuration for parallel dual-site control.

For each DOF, a pair of antagonistic muscles was used to control output velocity. The differences in EMG signal amplitude for the two muscles were calculated after each amplitude was conditioned by a linear gain and threshold. The muscles used included pronator teres (PT), supinator (SUP), flexor carpi radialis (FCR), extensor carpi radialis longus (ECRL), flexor digitorum profundus (FDP), and extensor digitorum communis (EDC).

$$\%\text{Velocity Activation}=\{{\mathrm{G}}_1{\mathrm{M}}_1-{\mathrm{T}}_1\}-\{{\mathrm{G}}_2{\mathrm{M}}_2-{\mathrm{T}}_2\}$$ (1)

where M₁ is the MAV of the first muscle in the antagonistic pair, G₁ is the gain applied, and T₁, is the threshold. Similarly, M₂, G₂, and T₂ represent the MAV, gain, and threshold for the second muscle. {•} represents max(0, •).

All gains and thresholds were manually set to minimize unintended DOF activity, to maximize the dynamic range of velocities, and so that 50% of the maximum possible velocity was achieved through comfortable-intensity contractions. A post-processing velocity-dependent ramp that was previously developed for pattern recognition control was also implemented for each DOF (Simon et al., 2011) to ensure similar post-processing conditions between the two control methods. The ramp caused the output velocity to linearly increase to 100% of the intended velocity, calculated by Eq. 1, over 500 ms of continuous activity in the predicted motion class.

2.2.2. Pattern Recognition Control

A myoelectric pattern recognition “proportional mutex” control scheme was implemented (Fougner et al., 2012, Englehart and Hudgins, 2003) for the following seven motion classes: pronation and supination; wrist flexion and extension; hand open and close; and no motion. The classifier was trained using 12 s of self-selected, comfortable-intensity contractions (four repetitions of 3 s contractions) for each motion class. EMG signals were partitioned into 250 ms windows with a 50 ms frame increment (Smith et al., 2011), from which four time-domain features (mean absolute value, number of zero crossings, waveform length, and number of slope sign changes (Hudgins et al., 1993)) and the coefficients of a sixth-order autoregressive model (Huang et al., 2005) were calculated for each EMG channel. A linear discriminant analysis classifier that has been shown to perform similarly to other classification methods for the movements investigated in this study (Hargrove et al, 2007) was used for ease of implementation. The average classification error for each subject was assessed offline by fourfold cross validation of the training data set. Classification error was calculated by dividing the number of erroneous class decisions by the total number of decisions made.

During real-time control, the output velocity was calculated proportional to the MAV of the EMG, and normalized by class-specific MAV averages calculated from the training data (Scheme et al., 2013):

$$\%\text{Velocity Activation}=\mathrm{G}{\left(\frac{\mathrm{M}·{\mathrm{M}}_{\mathrm{j}}^{\text{Tr}}}{{\mathrm{M}}_{\mathrm{j}}^{\text{Tr}}·{\mathrm{M}}_{\mathrm{j}}^{\text{Tr}}}\right)}^2-\mathrm{T}$$ (2)

where M is a vector of the current window's MAVs, and ${\mathrm{M}}_{\mathrm{j}}^{\text{Tr}}$ is the a vector containing the average MAV values for the training EMG signals labeled as motion class j. G and T are a gain and threshold set empirically from previous experiments to map the comfortable-level contractions produced during training to middle-range velocities. A post-processing velocity dependent ramp (Simon et al., 2011) was also implemented to reduce the effects of spurious misclassifications on performance.

2.3. Fitts' Law Test

A pseudo three-dimensional Fitts' Law task as described by Scheme and Englehart (2012) was used to evaluate subjects' real-time control of three DOFs. Fitts originally described a center-out target acquisition task in 1954, which was used to model the trade-off between speed and accuracy (Fitts, 1954), and has since been used to quantify controllability in human-computer interfaces (Williams and Kirsch, 2008, MacKenzie, 1992). In this experiment, subjects used the control systems to manipulate the position and size of a ring cursor into annulus-shaped targets (Fig. 2A). Wrist flexion/extension controlled horizontal cursor movement, supination/pronation controlled vertical cursor movement, and hand open/close controlled the cursor's radius (Fig. 2B). Trial success required the cursor to dwell within the target for 2 s. Trials in which this dwell was not achieved within 30 s, or in which the cursor overshot the target (crossed the target boundary) five times were regarded as unsuccessful. Subjects were neither encouraged nor discouraged from using simultaneous control for the parallel dual-site condition but were told that movements could be controlled simultaneously if desired.

Fig. 2. Fitts' Law Task.

(A) Experimental setup. Able-bodied subjects controlled a ring-shaped cursor in a virtual environment. Subjects were restrained in a forearm splint to produce isometric contractions. (B) Ring cursor control mapping. Subjects controlled the ring cursor using an intuitive mapping originally proposed by Scheme and Englehart (2012). Wrist flexion/extension controlled horizontal displacement, pronation/supination controlled vertical displacement, and hand open/close controlled the radius of the ring. (C) Target complexities. Subjects were instructed to move the ring cursor into a variety of annulus-shaped targets. Three different levels of target complexity were presented to subjects. 1-DOF targets required use of only one DOF to complete the task (e.g. wrist extension). 2-DOF targets required use of two DOFs to complete the task (e.g. wrist extension and supination). 3-DOF targets required use of all three DOFs to complete the task (e.g. wrist extension, supination, and hand open). (D) Relationship between two-dimensional ring task-space and a three-dimensional workspace. The location and radius of each target in the ring task space corresponded to a target location in a corresponding three-dimensional workspace from which target distance was calculated (Eq. 3). The path numbers shown (1-3) correspond to movements depicted in (C).

Targets locations in the pseudo three-dimensional workspace were randomly sampled from a predetermined set of locations, including on- and off-diagonal combinations of the DOFs. Three levels of target complexity were included in this set: “1-DOF targets” required activity in only one DOF, “2-DOF targets” required use of two DOFs, and “3-DOF targets” required use of all three DOFs to successfully complete the task (Fig. 2C). Each of the three target complexities were presented to the subjects with equal frequency.

For each target complexity, six combinations of annulus thicknesses (W) and distance (D) were presented to the subjects (Table 1). Target annulus distance was calculated in the three-dimensional Cartesian workspace depicted in Figs. 2C-D:

Table 1.

Target conditions of the Fitts' Law test.

Distance, D (normalized distance unit)	Thickness, W (normalized distance unit)	Index of Difficulty, ID (bits)
40	10	2.32
40	20	1.59
40	30	1.22
80	10	3.17
80	20	2.32
80	30	1.87

$$\mathrm{D}=\sqrt{{\mathrm{X}}^2+{\mathrm{Y}}^2+{\mathrm{R}}^2}$$ (3)

where X is the horizontal translational distance, Y is the vertical translational distance, and R is the difference in radii between the cursor starting position and the target. The index of difficulty of a given combination of annulus thickness and distance was calculated as defined by MacKenzie (1992):

$$\text{ID}={log}_2(\frac{\mathrm{D}}{\mathrm{W}}+1)$$ (4)

Data were collected over six experimental blocks separated by rest periods. Each block had three repetitions for each possible combination of the six indices of difficulty and three target complexities, resulting in 54 targets per block. Performance metrics for the Fitts' Law test were throughput and path efficiency. Throughput (TP) was defined as calculated by the mean of means method suggested in ISO 9241-9 (Soukoreff and MacKenzie, 2004)

$$\text{TP}=\frac{1}{\mathrm{N}}\sum _{\mathrm{i}=1}^{\mathrm{N}}\frac{{\text{ID}}_{\mathrm{i}}}{{\text{MT}}_{\mathrm{i}}}$$ (5)

where N is the number of target conditions and MT is the mean time to acquire the target condition. Path efficiency (PE) was defined as by Williams and Kirsch (2008) as:

$$\text{PE}=100\%\ast \frac{\text{straightline}\_\text{distance}}{\text{distance}\_\text{taken}}$$ (6)

For parallel dual-site control, use of simultaneous control was characterized by identifying the number of EMG windows that contained simultaneous activity of two or three DOFs. Simultaneous control activity was calculated both as a fraction of the entire trial duration and as a fraction of the trial partitioned into ten deciles. Subject learning was evaluated by calculating the degree of simultaneous DOF activity, throughput, and path efficiency for each experimental block.

2.4. Motion Test

A modified Motion Test (Kuiken et al., 2009) was implemented. In this task, subjects were prompted to perform and hold one of six isolated 1-DOF movements (e.g. supination) with visual feedback (Fig. 3). A successful trial required the control system to output the prompted 1-DOF movement sixty times within 10 s. Instances where unintended DOFs were activated in addition to the target movement were not counted towards the sixty successful outputs. Each of the six possible 1-DOF movements (pronation and supination; wrist flexion and wrist extension; hand open and close) were evaluated in blocks of three repetitions. Four complete sets of the Motion Test were executed with rest periods in between, resulting in twelve repetitions of each of the six target movements. Performance metrics including completion rate and average time to task completion were determined for each of the six target movements.

Fig. 3. Motion Test Graphic Interface.

Subjects were provided a visual prompt indicating a desired 1-DOF movement (right). Subjects were also provided visual feedback reflecting what the output of the myoelectric control system (left). Subjects were required to isolate the prompted movement and hold until the control system outputted the prompted 1-DOF movement sixty times.

2.5. Statistical Analysis

Linear regression with a t-test of the regression coefficient was used to evaluate whether simultaneous DOF activity, throughput, or path efficiency showed statistically significant changes as the experiment progressed. Differences between the Fitts' Law performance metrics at the end of the experimental session were evaluated for each target complexity using paired t-tests. Differences between the Motion Test performance metrics was also evaluated using paired t-tests. Significance was evaluated at α = 0.05. Averages are reported ± standard error of the mean (SEM).

3. Results

3.1. Control System Configuration

Both pattern recognition control and parallel dual-site control were successfully configured to the subject's preferred settings in approximately 30 minutes. Average classification error for the pattern recognition classifiers was very low at 0.23% ± 0.067%.

Fig. 4 demonstrates a typical reconstruction of the output of the two myoelectric control systems. Using parallel dual-site control, subjects were able to control the three DOFs simultaneously. Different velocities profiles were seen for each DOF, reflecting independent proportional control. However, the pattern recognition control restricted subjects controlling only one DOF at a time.

Fig. 4. Representative reconstructions of myoelectric control system outputs.

Both signals were recorded as a subject acquired a 3-DOF target in the Fitts' Law task. (A) Parallel dual-site control. Subjects used physiologically appropriate antagonistic muscle pairs to control each DOF. The MAV of the EMG signals from each antagonistic muscle pair were amplified and thresholded, and the difference provided the velocity output of the DOF. Subjects independently modulated the forearm muscles to simultaneously activate multiple DOFs simultaneously. The muscles used included pronator teres (PT), supinator (SUP), flexor carpi radialis (FCR), extensor carpi radialis longus (ECRL), flexor digitorum profundus (FDP), and extensor digitorum communis (EDC). (B) Pattern recognition control. Features calculated from each of the six muscles were used as input into a linear discriminant analysis classifier. The trained classifier predicted output of the intended motion class, which was restricted to controlling one DOF at a time.

3.2. Fitts' Law Task: Myoelectric Control Performance during a Virtual Task

On average, subjects were able to complete 99.9% of all Fitts' Law trials within the time limit and without an overshoot penalty. Mean completion time varied linearly with index of difficulty for each control scheme and target complexity (Fig. 5). The r² values for the linear regression models ranged between 0.919 and 0.996, indicating that a Fitts' Law analysis was appropriate.

Fig. 5. Empirical Fitts' Law regression models.

Relationship between completion time versus index of difficulty for both control systems. Data was averaged across all subjects for all six experimental blocks. (A) Parallel dual-site control. r² for 1-, 2-, and 3-DOF targets was 0.919, 0.961, and 0.976, respectively. (B) Pattern recognition control. r² for 1-, 2-, and 3-DOF targets was 0.939, 0.976, and 0.996, respectively.

When subjects used parallel dual-site control for tasks requiring multiple DOFs, throughput increased during the experimental session (Fig. 6A). For 2-DOF targets, throughput significantly increased with each successive experimental block (regression coefficient t-test, p<0.05). The average throughput was 0.81 ± 0.09 (standard error of the mean, SEM) bits/s in block 1 and 1.06 ± 0.07 bits/s in block 6. For 3-DOF targets, throughput also significantly increased with number of experimental blocks completed (regression coefficient t-test, p<0.05). Average throughput was 0.56 ± 0.04 bits/s in block 1 and 0.71 ± 0.05 bits/s in block 6. In contrast, when using pattern recognition control, throughput showed no significant or consistent change during the course of the experiment for 2- or 3-DOF targets. Average throughput was 0.91 ± 0.06 bits/s for 2-DOF targets and 0.57 ± 0.04 bits/s for 3-DOF targets. During the last experimental block, throughput was 10% and 26% greater for 2- and 3-DOF targets, respectively, for parallel dual-site control compared to pattern recognition control (Fig. 6B). This increase was statistically significant for the 3-DOF targets (paired t-test, p<0.05). There was no significant time-dependent change in throughput for 1-DOF targets using either control method and no statistical difference in throughput between the control methods for 1-DOF targets during the last experimental block. Average throughput for 1-DOF targets was 2.24 ± 0.10 bits/s for parallel dual-site control and 2.35 ± 0.12 bits/s for pattern recognition control.

Fig. 6. Throughput measures during the Fitts' Law test targets requiring use of 1-, 2- or 3-DOFs.

(A) Changes as the experimental sessions progressed. For 1-DOF targets, no significant trend was observed in throughput for either control system. For targets requiring use of 2- or 3-DOFs, throughput increased throughout the course of the experiment when subjects used parallel dual-site control (p<0.05) but remained stable throughout the experiment for pattern recognition control. Note changes in y-axis. (B) Comparison during last experimental block. In the last session of the experiment, throughput in 2- and 3-DOF targets was greater for parallel dual-site control than for pattern recognition control. When using parallel dual-site control, throughput was 10% and 26% greater for 2- and 3-DOF targets, respectively, than when using pattern recognition control (p<0.05, for 3-DOF targets). Error bars represent mean ± SEM.

Path efficiency for 2- and 3-DOF tasks also increased during the experimental session when using parallel dual-site control (Fig. 7A). This increase was significant for 2-DOF targets (regression coefficient t-test, p<0.05), where subjects started with 63.6% ± 3.3% efficiency in experimental block 1 and ended with 73.1% ± 2.8% efficiency in block 6. In contrast, when using pattern recognition control, path efficiency showed no significant change over the experimental session for 2- or 3-DOF target complexities. Average efficiency for pattern recognition control was 67.4% ± 0.7% for 2-DOF targets and 54.5% ± 1.2% for 3-DOF targets. During the last experimental block, path efficiency was 9% and 5% greater for 2- and 3-DOF targets, respectively, for parallel dual-site control compared to pattern recognition control, though neither increase was statistically significant (Fig. 7B). Of note, the theoretical maximum efficiency of acquiring an average 2-DOF or 3-DOF target using pattern recognition control was 73.6% and 63.8%, respectively. These theoretical maxima are less than 100% because pattern recognition enforces sequential control of DOFs. No statistically significant change in path efficiency was observed over the experimental sessions for 1-DOF targets using either control method and there was no statistical difference in path efficiency between control methods for 1-DOF targets during the last experimental block. Average path efficiency for 1-DOF targets was very high at 97.1% ± 6.3% for parallel dual-site control and 98.1% ± 6.4% for pattern recognition control.

Fig. 7. Path efficiency measures during the Fitts' Law test targets requiring use of 1-, 2- or 3-DOFs.

(A) Changes as the experimental sessions progressed. For 1-DOF targets, no significant trend was observed in path efficiency for either control system. For targets requiring use of 2- or 3-DOFs, efficiency increased throughout the course of the experiment when subjects used parallel dual-site control (p<0.05 for 2-DOF target) but remained stable throughout the experiment for pattern recognition control. The black dashed lines represents the theoretical maximum efficiency that could be obtained using pattern recognition control. For the 2-DOF targets, subjects exceeded this maximum when using parallel dual-site control. Note changes in y-axis. (B) Comparison during last experimental block. In the last session of the experiment, efficiency in 2- and 3-DOF targets was greater for parallel dual-site control than for pattern recognition control, though these changes were not statistically significant. Error bars represent mean+SEM.

3.3. Fitts' Law Task: Use of Simultaneous Control

Subjects frequently used simultaneous control to complete both 2- and 3-DOF targets with the parallel dual-site method (Fig. 8). For the more complex 3-DOF targets, subjects used significantly more simultaneous control during later experimental blocks (regression coefficient t-test, p<0.05). In block 6, simultaneous control was used during 49.7% ± 4.7% of the trial duration for 2-DOF targets and 50.3% ± 3.6% of the trial duration for 3-DOF targets. In contrast, on average, simultaneous control was used for 6.2% ± 1.4% of the trial for 1-DOF targets for the duration of the experiment.

Fig. 8. Changes in use of simultaneous control as the experimental sessions progressed.

Subjects used simultaneous control extensively for 2- and 3-DOF targets. As the experiment progressed, subjects used significantly more simultaneous control for the 3-DOF targets (p<0.05). Data are averaged across all subjects. Error bars represent mean ± SEM.

Fig. 9 shows the subjects' simultaneous control of DOFs using parallel dual-site control in the last experimental block. For 1-DOF targets, subjects mostly used only one DOF, with a steady low level of 2-DOF activation throughout the trial duration. For 2- and 3-DOF targets, the degree of simultaneous DOF activation varied throughout the progression of the trial. Subjects started trials using 1-DOF activation and gradually introduced simultaneous control of additional DOFs. Maximum simultaneous activation occurred in the middle of the trial, with higher levels of 1-DOF activation again observed at the end of the trial. Subjects simultaneously activated all three DOFs most often during trials with 3-DOF targets, whereas there was a steady low level or no 3-DOF activity for 1- and 2-DOF targets.

Fig. 9. Averaged use of sequential vs simultaneous control during a Fitts' Law trial, experimental block 6.

For 1-DOF targets, subjects mostly activated one DOF but occasionally activated two DOFs. For 2-DOF targets, subjects often began the trial using only one DOF and gradually added simultaneous activation of a second DOF. Towards the end of the trial, subjects were again more likely to use only one DOF at a time. For 3-DOF targets, as for 2- DOF targets, subjects were more likely to use simultaneous control during the middle of trial and were more likely to use one DOF at a time at the beginning and the end. Horizontal axis represents normalized trial duration, partitioned into deciles. Vertical axis represents the distribution of 1-DOF and simultaneous 2- and 3-DOF activity during a given decile for targets of different complexities. Data were averaged across all subjects. Shaded regions represent mean ± SEM.

Fig. 10 shows sample cursor trajectories for the two control methods and depicts how using simultaneous control affected trajectory characteristics. For the parallel dual-site method, subjects generated curved trajectories as they simultaneously activated multiple DOFs to acquire 2- and 3-DOF targets. These trajectories differed greatly from the trajectories generated by the sequential DOF activation enforced by pattern recognition control, which resulted in sharp orthogonal changes in direction as the intended motion class changed. When using the parallel dual-site method, subjects also occasionally activated simultaneous-DOF activity when completing 1-DOF targets. When using pattern recognition control, off-axis deviations for 1-DOF targets were orthogonal and represented misclassifications. Videos reconstructing the most and least efficient trials for three subjects using each of the two control methods can be found in the supplemental materials online (Movies S1-S3).

Fig. 10. Representative target acquisition paths for parallel dual-site and pattern recognition control.

Paths are presented in three-dimensional workspace, described in more detail in Fig. 2. Each plot shows a within-subject comparison between parallel dual-site (red) and pattern recognition (blue) control. Targets are represented as grey spheres. Black hashed lines represent straight-paths to the target (100% path efficiency). (A) Samples from trials with highest recorded path efficiency. When using parallel dual-site control, subjects were able to simultaneously control multiple DOFs when acquiring the target. Curved trajectories resulted, with the potential to follow closely to a path with perfect path efficiency. In contrast, when using pattern recognition control, paths show sharp, orthogonal changes in direction as subjects change which DOF they are using. (B) Samples from trials with lowest recorded path efficiency. When using parallel dual-site control, lower-efficiency paths frequently resulted from unintended movement in DOFs, as depicted by the off-axis movement in the 1-DOF target. Subjects also occasionally overshot the target when using simultaneous control (2- and 3-DOF targets). In contrast, when using pattern recognition, lower-efficiency paths resulted from misclassifications of the intended motion class. Videos depicted the trials in the ring-task interface are found in Movies S1-S3.

3.4. Motion Test: Isolating Movements Requiring Only One DOF

On average, subjects successfully completed 98.3% of the trials when using parallel dual-site control, compared to 100% of the targets when using pattern recognition. Subjects successfully completed the task on average within 3.64 ± 0.28 s when using the parallel dual-site control method, compared to 3.14 ± 0.10 s when using pattern recognition. The averaged differences between parallel dual-site control and pattern recognition were not statistically significant.

The Motion Test results, when broken down by target motion, showed more performance variability when subjects used parallel dual-site control than pattern recognition control (Fig. 11). Subjects occasionally had trouble isolating one of the six possible movements when using the parallel dual-site method, i.e., additional DOFs were unintentionally activated. These difficulties resulted in longer trial completion times and greater within-subject variability compared to pattern recognition control, which did not allow subjects to activate more than one DOF at a time.

Fig. 11. Distribution of Motion Test completion times for isolated 1-DOF movements.

There was no statistically significant difference between the averaged Motion Test performance of the two control systems. However, the within-subject spread of completion times were substantially different between the two control systems. Subjects occasionally had trouble isolating one of the six possible movements when using parallel dual-site control, leading to longer completion time for those specific trials and increased within-subject variability. Box plots represent the distribution of mean completion times for each 1-DOF movement. (+) represents data points outside 1.5× the interquartile range.

4. Discussion

Our data suggest that intramuscular EMG, when used with a parallel dual-site control paradigm, has the potential to provide simultaneous, proportional control of a multi-DOF prosthetic wrist-hand system. This capability would enhance the control and functionality of myoelectric upper limb prostheses, which are currently limited by methods that impose sequential control of movements. Most previous investigations of simultaneous control have focused on offline analyses with able-bodied subjects. This study demonstrated simultaneous control of three DOFs in real time (Fig. 10, Movies S1-S3), where subjects can independently and proportionally modulate the velocity of each DOF. This work constitutes a necessary step toward the evaluation of simultaneous control algorithms in amputees and eventual clinical implementation. The parallel dual-site control approach has been frequently proposed for use with implantable myoelectric recording devices (Weir et al., 2009, Merrill et al., 2011, Birdwell, 2012, Tucker and Peteleski, 1977, Stein et al., 1980), many of which are now in development (Weir et al., 2009, Young, 2009, Bercich et al., 2012, McDonnall et al., 2012).

Multiple repetitions of the Fitts' Law trial were used to characterize subjects' learning process. For each control method, subjects performed six blocks of the Fitts' Law trials. As the trials progressed, subjects showed significant performance metric improvements when using parallel dual-site control in trials that required use of two or more DOFs (Figs. 6A and 7A). The increased use of simultaneous control as the experiment progressed (Fig. 8), together with the minimal learning observed for 1-DOF targets (Figs. 6A and 7A), suggest that performance improved as subjects learned to use the simultaneous control. As subjects learned to use parallel dual-site control, achievement of 2- and 3-DOF targets in the Fitts' Law task gradually surpassed performance using sequential pattern recognition control (Figs. 6A and 7A). Previous studies (Bunderson and Kuiken, 2012, Powell et al., 2013) have also demonstrated the potential for learning while using sequential pattern recognition control, yet subjects in this study showed no evidence of learning when using sequential pattern recognition. Subjects' pattern classifiers had very low classification error rates (< 1% error in an offline cross-validated analysis), suggesting that the steady-performance observed may reflect subjects' maximum attainable performance using this control method. This was expected, as all subjects had previous experience using pattern recognition control with surface EMG.

The lack of a performance plateau over time with the parallel dual-site system suggested that performance may have continued to improve given additional trials and/or more days of practice. Our study was limited by the duration of the experiment; six trial blocks resulted in experimental sessions lasting approximately 5 h, which was considered an upper limit for subject attention span and comfort. The potential for continuing improvement with more practice is compelling. Future studies are required to follow learning as subjects use these control methods over multiple days.

By the end of six experimental blocks, statistically significant differences in performance between the two control methods began to emerge (Figs. 6B and 7B). Notably, throughput was 25% greater for the 3-DOF targets when using parallel dual-site control compared to pattern recognition (p<0.05). Though improvements in path efficiency fell short of statistical significance, these increases are intriguing in light of the small sample size and continued learning effect and warrant further investigation. Importantly, although only modest efficiency improvements were seen for 2- and 3-DOF targets, these differences allowed parallel dual-site control to approach, and at times exceed, the maximum possible efficiencies attainable for sequential pattern recognition control (Fig. 7A). In contrast, the average path efficiency executed with pattern recognition held steady at values lower than these theoretical maxima.

The results of this study therefore suggest that intramuscular EMG, when used in a parallel dual-site control system, has potential to provide simultaneous control of a 3-DOF prosthetic wrist/hand system that surpasses the functionality of current sequential-control algorithms. These results are promising, and correspond with previous studies demonstrating the performance benefit of simultaneous myoelectric control in 2-DOF systems (Wurth and Hargrove, 2013, Jiang et al., 2013). Further experimentation is needed to evaluate how the results of this study extend to simultaneous control of DOFs in a physical prosthesis.

When using dual-site parallel control, subjects voluntarily chose to use simultaneous control without explicit prompting. Previous studies have shown mixed results regarding the voluntary use of simultaneous control in EMG-based virtual tasks. Williams and Kirsch implemented a system similar to parallel dual-site control using head/neck muscles in patients with tetraplegia (Williams and Kirsch, 2008), but found that subjects chose not to control DOFs simultaneously in a 2-DOF Fitts' Law task. Birdwell found that subjects using parallel dual-site control with EMG from extrinsic finger muscles activated two of three DOFs simultaneously in a virtual task (Birdwell, 2012). However, subjects in the current study controlled all three DOFs at the same time (Figs. 4A, 9, and 10). Our subjects performed over 600% more trials than in Birdwell's study; this suggests that the amount of practice, ease of activation, and/or the physiological relevance of the muscles used for prosthesis control may influence the choice to use simultaneous control. As in Birdwell's study on finger control, subjects in this study followed a characteristic pattern of simultaneous DOF activation for the wrist and hand. An increase in simultaneous DOF activity during the first half of the trial corresponded with subjects' gross positioning of the cursor, whereas the high percentage of sequential-DOF control at the end of the trial reflected subjects' fine, corrective movements.

A potential limitation of simultaneous myoelectric control is the possibility of unintended additional movement when trying to perform isolated movement of one DOF. However, for both parallel dual-site and sequential pattern recognition control, subjects were able acquire 1-DOF Fitts' Law targets with averaged path efficiencies of greater than 95% and to complete the Motion Test with an average time of less than 4 s (3 s is the fastest possible completion time). Though no statistical difference was observed between the average performance of the two control systems in the Motion Test, the within-subject spread of completion times was different (Fig. 11). The Motion Test results suggested that some subjects had difficulty in isolating one or two of the six possible 1-DOF motions, but could isolate the other motions well. The additional, unintended DOF activation was not frequent or extensive enough to greatly affect subjects' averaged performance, but subjects reported that it was frustrating.

Difficulty in isolating DOFs likely resulted from inability to isolate muscle contractions during tasks, given the small pick-up radius of intramuscular electrodes (Basmajian and De Luca, 1985) and that subjects were able to isolate EMG channels when the limb was supported during fine wire insertion. Similar difficulty was described in a transradial amputee using parallel dual-site control (Stein et al., 1980), but was reported to improve with practice over multiple weeks. Unintentional activation did not significantly decrease over the course of this experiment, though a longer experiment and/or more days of practice may have reduced unintended activity. Unintended activation during parallel dual-site control was also affected by gains and thresholds applied to the EMG signals, which were manually set at the beginning of the experiment. These parameters could be modified as subjects gain experience. Ideally, a myoelectric control system should incorporate the positive attributes of the two control systems evaluated in this study: the ability to simultaneously activate DOFs (as in parallel dual-site control), and the ability to isolate a DOF for single-dimensional tasks as desired (as in pattern recognition control). Future work should therefore compare the real-time performance of parallel dual-site control with control methods that do not require subjects to independently control individual muscles, such as simultaneous pattern recognition (Young et al., 2012, Kamavuako et al., 2013a) or continuous predictions of joint kinematics or torques using neural-networks (Kamavuako et al., 2012, Jiang et al., 2012). Real-time control of a multi-DOF wrist-hand system has not been evaluated using these methods. Targeted intramuscular EMG should also be considered as a signal source for these approaches as access to deeper muscles and reduction in crosstalk may allow for improved control. Furthermore, each of these control systems requires extensive sets of training data to account for all possible DOF combinations. A chronically implanted electrode may allow subjects to retrain such systems less frequently than when using surface EMG, where electrode shift or changes in skin impedance can lead to deterioration of controllability. Future experiments should also include transradial amputee subjects, who are the target users for such a control system. Parallel dual-site control is limited to individuals with a transradial or lower level amputation, such that physiologically appropriate muscles can be used to control prosthetic DOFs.

5. Conclusion

Our study demonstrates that use of intramuscular EMG can provide simultaneous control of multiple DOFs in real-time, which is an important next step in improving the control of powered prosthetic arms.