Multigrasp Myoelectric Control for a Transradial Prosthesis

Abstract

This paper presents the design and preliminary experimental verification of a multigrasp myoelectric controller. The controller enables the direct and proportional control of a multigrasp transradial prosthesis through a set of nine postures using two surface EMG electrodes. Five healthy subjects utilized the multigrasp controller to manipulate a virtual prosthesis to experimentally quantify the performance of the controller in terms of real time performance metrics. For comparison, the performance of the native hand was also characterized using a dataglove. The average overall transition times for the multigrasp myoelectric controller and the native hand with the dataglove were found to be 1.49 and 0.81 seconds, respectively. The transition rates for both were found to be the same (99.2%).

I. Introduction

Prosthetic restoration of the native hand’s functionality and appearance remains a considerable challenge. Prior limitations in battery energy density, actuator size, and embedded electronics have restricted traditional commercially available myoelectric prostheses to single degree of freedom devices driven by a single actuator and commanded by a single electromyogram (EMG) input (two EMG electrodes placed on antagonistic muscle pairs of the residual forearm). This simplified approach does have its advantages, however. Primarily, controlling such devices requires little cognitive effort because of the one-to-one mapping between the actuator and EMG input. Also, traditional myoelectric prostheses can provide direct, proportional control of motion. Since they only require a single EMG input the interface is manageable, and is easily incorporated into a socket. Furthermore, the control method is not computationally demanding and can occur with minimal delay. A modern version of the single grasp myoelectric prosthesis is the MyoHand VariPlus Speed (Otto Bock).

Despite these advantages, single grasp devices have reduced grasping capability (because they cannot conform to objects and there is little contact area), and an unnatural appearance (due to their low level of articulation) [1]. This has been evidenced by patient surveys which indicate that increased articulation and enhanced appearance are among the top design priorities for users [2]. In response to this, and with the advancement of previously limiting technologies, several multigrasp prosthetic hands have been developed [2–9]. These multigrasp hands contain 8 to 16 joints, driven by 1 to 6 actuators. While increased complexity holds the potential for enhanced grasping capability and fidelity of motion, it requires the development of effective multigrasp control interfaces. It has been stated that the lack of more sophisticated hand prostheses is primarily due to absence of effective control systems [10].

An effective multigrasp controller must enable the user to access the multifunctional capability of the hand dependably and efficiently. This should be done in a direct and intuitive manner which offers continuous and proportional control of motion with negligible latency. Prevalent approaches to multigrasp control thus far include pattern recognition [10–14] and hierarchical control [15–20].

Pattern recognition for multigrasp hands involves determining user intent (i.e. selecting postures and grasps) based on the observation of (often) a plurality of EMG inputs during a translating frame. In this approach, a classifier is trained which associates EMG input patterns with intended postures and grasps (i.e. classes). During operation, the classifier examines each frame of EMG data and makes a decision (or classification) about which class is being commanded.

Hierarchical control involves event-driven finite state machines. In this approach, sensory information is utilized to trigger events which cause transitions from state to state. The states represent stages of grasping or hand postures. Initially, hierarchical control was utilized for modulation of grasping within a given posture [15–17] (after a posture was selected the user could transition between the ‘touch’, ‘hold’, ‘squeeze’, and ‘release’ states, or return to the ‘position’ state). To solve the issue of posture selection, a hybrid controller was proposed in which the prehensile form (posture) would first be chosen using a pattern recognition algorithm followed by hierarchical control for the modulation of grasping [18, 19]. In [20],a hierarchical control scheme was used for posture selection itself, followed by automated grasping which could be initiated and terminated by the user. In this work state transitions depended on pattern recognition techniques.

This paper presents the design and experimental validation of a dependable, efficient and proportional controller for multigrasp transradial prostheses, referred to as Multigrasp Myoelectric Control (MMC). User input to the MMC consists of two surface EMG electrodes; an interface configuration which has proven effective for the control of single degree of freedom prosthetic devices. This approach is enabled by the use of position and force sensing in the hand prosthesis, which is utilized in the presented control structure to provide low-level coordination of grasping and hand postures. In this way the input (e.g., EMG) from the user need only convey to the prosthesis high-level information regarding direction of motion and degree of effort. This paper provides a detailed description of the MMC (Section II), followed by a presentation of experimental methods (Section III). The experimental results are then discussed (Section IV), and the papers closes with conclusions and future work (Section V).

II. Multigrasp Myoelectric Control

At its core, the MMC consists of an event driven finite-state machine (FSM) represented by the state chart in Fig. 1. The state chart consists of a finite set of states (represented in the figure by photographs), the transitions amongst these states (represented in the figure with arrows), and the conditions and events upon which these transitions depend. The future state of the prosthesis is determined as a function of the current state and the inputs to the FSM itself. The “states” in the MMC state machine consist of the postures and grasps which a multi-grasp hand may be capable of assuming. In the case of the Vanderbilt Multigrasp Hand (VMG Hand), whose design is described in [21], these states include: reposition (platform), point, hook, lateral pinch, opposition, tip, and cylinder/sphere/tripod (for this state, the grasp that the prosthesis actually assumes depends on the object being grasped, although the control actions for each grasp are the same). The prosthesis may transition from any given state (posture) to any other. However, the hand must follow a path consistent with the transition topology defined in the state chart. To accommodate the states described, the topography of the state chart has been constructed such that closing of the hand is associated with upward transitions in the state chart, while opening of the hand is associated with downward transitions in the state chart. Postures and grasps in which the thumb is opposed are located on the left side of the state chart, while postures and grasps in which the thumb is reposed are located on the right.

Fig. 1.

The structure of the MMC finite state machine. The states are depicted with photographs with the active tendons for each state indicated on the inset of each. Transitions from state to state depend on tendon excursions (given in mm), tendon forces, and twitch detection logic. Contraction of the forearm flexors is associated with upward movement in the state chart. Contraction of the forearm extensors is associated with downward movement in the state chart. Twitching initiates an automated toggling motion between the opposition and reposition states.

For a given state, a transition will occur if certain logical conditions have been met or certain events have transpired. These conditions and events are based on inputs to the state machine, chiefly: 1) Tendon Excursion (based on the position sensing of integrated hall-effect sensors on the brushless DC motors driving the tendons), 2) Tendon Force (based on the current sensing of the servo amplifiers driving the motors), and 3) Twitch Detection (based on twitch detection logic). For instance, in the point sate, if the tendon excursion or force for the index finger exceeds a certain threshold, then the hand will transition from the point to hook states. Twitching causes an automated toggling between reposition and opposition.

The output of the finite state machine, the current hand state (posture), governs which subset of motors is active in the hand at any given time. EMG input from the user only effects the active motors, and in this manner the MMC coordinates the movement of the prosthesis. For instance, if the user wishes to further close the hand, then they need only contract the anterior musculature of the forearm. This generates velocity level signals which are integrated and added to the positional references of the active motors. This is associated with upward movement along the state chart. Conversely, to open the hand, the posterior musculature of the forearm is contracted, and the velocity level signals which are generated are integrated and subtracted from the positional references of the active motors. This is associated with downward movement along the state chart. The speed with which the prosthesis opens or closes is proportional to the extent of muscular contraction. Relaxation of the forearm muscles halts prosthesis movement.

As the digits of the prosthesis move or interact with objects in the environment, transitions in the state chart may occur. (Twitching may also initiate state transitions.) Once a transition occurs, the current state of the hand changes, and a new subset of motor units become activated by the coordination controller. The motor units which are active are always those associated with transitions to adjacent states. This configuration allows for direct, continuous motion between states (the exception being the automated transition between opposition/reposition).

III. Experimental Methods

A. Subject Information and Testing Overview

Five healthy (non-amputee) subjects aged 22–44 participated in this study. Of these subjects, 4 were right-handed and one was left-handed. Each subject participated in six trials which involved using both the dataglove and the MMC to control the movement of a virtual prosthesis. (see Fig. 2) The time between trials ranged from one day to three weeks and all subjects completed the six experimental trials within a time frame of approximately one month. The experimental protocol for this study was approved by the Vanderbilt University Institutional Review Board.

Fig. 2.

The virtual prosthesis (light) and virtual ghost (dark). The virtual ghost is controlled automatically by the computer, and serves as a postural reference for the user. When the user brings the virtual prosthesis sufficiently close to the virtual ghost, the ghost is no longer displayed, indicating to the user that they have acquired the desired target posture.

B. Data Acquisition with Dataglove

A dataglove was constructed to track the movement of the native hand. This was accomplished using variable resistance flex sensors (Spectra Symbol) attached to a lightweight, highly flexible, slip-on glove (Fox Head, Inc.). Four flex sensors were used to capture the flexion and extension of the first, second, and third digits (i.e. the thumb, index finger, and middle finger) as well as thumb opposition. Note that the fourth and fifth digits were assumed to track the third (i.e. the ring and little fingers were assumed to follow the middle finger). This allowed for the detection of the postures attainable with the configuration of the MMC described in this paper. The variable resistance flex sensors were incorporated into a voltage divider circuit whose output was amplified, adjusted, and buffered before being acquired using a data acquisition card (Humusoft MF624) and accessed using Simulink Real Time Windows Target (The Mathworks). These signals were then low-pass filtered and normalized in Simulink to be used as position references for control of the virtual prosthesis.

C. Data Acquisition with EMG

Two self-adhesive, Ag/AgCl, snap bipolar electrodes with a spacing of 22 mm (Myotronics, Inc.) were used for EMG data acquisition. The electrodes were placed on the anterior and posterior surfaces of the subject’s forearm in the approximate vicinity of the flexor carpi radialis (Electrode 1) and extensor carpi radialis muscles (Electrode 2) after cleaning each site with an alcohol pad. These analog signals were differentially preamplified with a gain of 100 and low pass filtered at a cutoff frequency of 500 Hz using custom analog circuitry at the electrode site. This information was sampled at 1 kHz using an MF624 data acquisition card (Humusoft) and accessed using Simulink Real Time Workshop (Mathworks). The signals were then digitally high-pass filtered at 50 Hz, rectified, and low pass filtered again for use as velocity references.

Before utilization for control, the EMG signals of each subject were characterized for normalization and twitch threshold identification. To do this, a Simulink model was used which visually prompted the user to go through a series of activities in real-time. These included: Flex (contraction of the flexor carpi radialis muscle), Extend (contraction of the extensor carpi radialis muscle), Twitch (co-contraction of the flexor and extensor muscles), Rest (relaxed state, no activity) and Disturb (the user was asked during this time to randomly move the wrist and digits, and to re-orient the arm being tested). All characterization data was collected in a single, automated, session lasting 210 seconds which occurred before the first experimental trial. The session consisted of three 60 second periods of alternating Flex and Rest, Extend and Rest, and Twitch and Rest activities, followed by 30 seconds of continuous Disturbance activity. The Flex, Extend and Rest activities were prompted for a random period of 2–5 seconds in each instance. The Twitch activity was prompted for 1 second in each instance. This resulted in approximately 8 instances each of Flex and Extend activity, approximately 12 instances of Twitch activity, and approximately 30 instances of Rest activity.

For the purpose of real-time normalization, mean values during Flex and Rest (Electrode 1) and Extend and Rest (Electrode 2) were established as upper and lower bounds for each signal, respectively. To enable real-time twitch detection, thresholds were established for each EMG channel based on an exhaustive search. A twitch was said to have occurred if both signals concurrently exceeded their respective thresholds. This process transformed the two EMG signals into two proportional, normalized signals which were used to provide velocity references and twitch events for multigrasp prosthesis control. This calibration was not repeated for the duration of the duration of the experimental trials.

D. Virtual Prosthesis

To create the virtual prosthesis (see Fig. 2), solid model part and assembly files of the hand prosthesis described in [21] were exported from Pro/Engineer (PTC) to a virtual reality modeling language file. The virtual ghost was created by inserting another, darkly shaded copy of the virtual prosthesis into the virtual environment. The virtual prosthesis and ghost were animated by signals generated in Simulink. The virtual prosthesis was controlled by the user via signals emanating from either the dataglove or MMC. The virtual ghost was controlled automatically by the computer and served as a postural reference for the user.

In this paper the MMC was modified to control the virtual prosthesis. Because the virtual environment did not allow for interaction with objects, force dependent transitions were not invoked in the state chart. Instead, state chart transitions were strictly position dependent. The velocity gains in the MMC were set so that the virtual prosthesis moved with speeds reflective of the capabilities of the VMG hand described in [21].

E. Experimental Procedure

Each trial consisted of two experimental tasks. The first task involved using the data glove to adopt a series of target postures with the virtual prosthesis (Fig. 2). This was done to characterize the performance of the native hand and thereby establish a performance benchmark for the MMC. In the second task the MMC was used to adopt a series of target postures with the virtual prosthesis.

The target postures (e.g. the reposition, point, hook, lateral pinch, opposition, tip, and cylindrical/spherical/tripod postures displayed in Fig. 1) were presented randomly. There exist 42 unique transitions among these seven states and each of these transitions was presented three times per task. This resulted in 126 movements per task, with each of the seven postures being presented 18 times.

Each target posture was displayed visually on a computer monitor by the virtual ghost. When a subject brought the virtual prosthesis to the target posture, the virtual ghost was no longer displayed, indicating that the user had achieved the desired pose. For a movement to be considered successful, and before a new target posture was displayed, it was required that the target posture be held for 3 seconds without excessive deviation (exceeding 25% of the total RoM for each digit). This was done both to deter overshoot and to allow subjects to rest between subsequent postures. Additionally, if a transition was not acquired within 5 seconds, it was considered unsuccessful and a new target posture was presented.

To prepare subjects for this procedure, the dataglove and MMC were explained beforehand. Each subject also used the dataglove and the MMC interface to control the virtual prosthesis until they were comfortable with the operation of each. In either case, and for all subjects, these familiarization periods required less than 5 minutes.

F. Performance Metrics

The time required for each transition, starting at the instant when the target posture was initially displayed and ending when the target posture was successfully acquired, was recorded. This was defined as the transition time. Note that the transition time as defined in this study includes visual, cognitive, neural, and muscular delay. The number of successfully completed transitions (those completed within 5 seconds) over the total number of attempted transitions was defined as the transition completion rate.

IV. Experimental Results & Discussion

A. Transition Times

The average transition times for the native hand and the MMC are given in Tables I and II, respectively, with standard deviations indicated in parenthesis. To provide a measure of overall performance, the average overall transition times were also calculated. This is the average time required to get to any given posture from any other. Average overall transition times for each subject, as well as the average overall transition time for all subjects are given in Table III. For the native hand, the average overall transition time for all subjects was 0.81 seconds with a standard deviation of 0.14 seconds. For the MMC the average overall transition time for all subjects was 1.49 seconds with a standard deviation of 0.15 seconds.

Table I.

Average Transition Times Of All Subjects Between Different Grasps and Postures for the Native Hand

		Target Posture
		Lateral	Hook	Point	Reposition	Opposition	Tip	Cyl/Sph/Tri
Original Posture	Lateral		0.81 (0.36)	0.81 (0.36)	0.79 (0.36)	0.76 (0.50)	0.78 (0.25)	0.74 (0.21)
	Hook	0.59 (0.12)		0.69 (0.25)	0.90 (0.65)	0.67 (0.19)	0.80 (0.28)	0.72 (0.19)
	Point	0.72 (0.31)	0.62 (0.15)		0.81 (0.60)	0.73 (0.19)	0.88 (0.26)	0.82 (0.28)
	Reposition	0.76 (0.25)	0.72 (0.18)	0.86 (0.48)		0.65 (0.28)	0.82 (0.50)	0.86 (0.22)
	Opposition	0.82 (0.30)	1.02 (0.48)	1.06 (0.53)	0.71 (0.19)		0.89 (0.30)	0.97 (0.38)
	Tip	0.77 (0.12)	0.94 (0.47)	0.90 (0.33)	0.96 (0.33)	0.71 (0.23)		0.91 (0.39)
	Cyl/Sph/Tri	0.85 (0.42)	0.92 (0.33)	0.86 (0.31)	0.86 (0.27)	0.72 (0.15)	0.84 (0.24)

Table II.

Average Transition Times of All Subjects Between Different Grasps and Postures for Multigrasp Myoelectric Control

		Target Posture
		Lateral	Hook	Point	Reposition	Opposition	Tip	Cyl/Sph/Tri
Original Posture	Lateral		1.20 (0.63)	1.32 (0.53)	1.37 (0.21)	2.02 (0.70)	2.43 (0.68)	2.70 (0.69)
	Hook	0.67 (0.14)		0.89 (0.29)	1.05 (0.14)	1.60 (0.39)	2.03 (0.51)	2.50 (0.95)
	Point	1.12 (0.35)	0.84 (0.22)		0.81 (0.43)	1.25 (0.36)	1.67 (0.32)	2.21 (0.50)
	Reposition	1.82 (0.96)	1.34 (0.36)	1.02 (0.31)		0.92 (0.51)	1.36 (0.51)	1.57 (0.56)
	Opposition	1.84 (0.44)	1.79 (0.56)	1.38 (0.43)	0.75 (0.26)		1.11 (0.43)	1.47 (0.88)
	Tip	2.16 (0.44)	2.18 (0.56)	1.68 (0.38)	1.15 (0.44)	0.61 (0.13)		0.75 (0.10)
	Cyl/Sph/Tri	2.40 (0.40)	2.46 (0.61)	1.97 (0.55)	1.40 (0.37)	0.88 (0.17)	0.85 (0.22)

Table III.

Average Overall Transition Times and Transition Rates

Subject	Criteria	Dataglove	MMC
HS1 (RH)	Transition Time	0.74	1.48
	Transition Rate	99.2	99.7
HS2 (LH)	Transition Time	1.04	1.73
	Transition Rate	99.2	98.1
HS3 (RH)	Transition Time	0.84	1.50
	Transition Rate	99.2	99.7
HS4 (RH)	Transition Time	0.75	1.33
	Transition Rate	99.2	99.2
HS5 (RH)	Transition Time	0.68	1.40
	Transition Rate	99.2	99.5
Average	Transition Time	0.81 (0.14)	1.49 (0.15)
	Transition Rate	99.2 (0.00)	99.2 (0.67)

RH and LH are abbreviations for right handed and left handed, respectively. The standard deviations are given in parentheses.

As can be seen in Table I, transition times for the native hand are relatively uniform (standard deviation of 0.10 seconds) when compared to the MMC (standard deviation of 0.58 seconds). However, the distribution of transition times for the MMC is related to the distance between the original and target postures on the state chart. That is, transition times between adjacent postures are relatively short, whereas transition times between states which lie at the far ends of the state chart (e.g. the lateral pinch and cylinder/sphere/tripod grasps) are longer.

B. Transition Completion Rate

The average transition completion rates for each subject, as well as the average transition completion rate for all subjects are given in Table II, with standard deviations in parenthesis. The transition rate for the data glove was 99.2% with a standard deviation of 0.00% (all subjects had 3 failed attempts when using the dataglove). The average transition rate for the MMC was also 99.2%, but with a standard deviation of 0.67% (subjects had from one to seven failed attempts using the MMC). Qualitative subject feedback indicated that causes for missed transitions included misinterpretation of reference postures, accidental excessive deviation from the target posture, and the inability to twitch.

These data indicate that transitions were completed as often with the MMC as with the native hand and that transitions were completed in less than 5 seconds nearly 100% of the time. Because of this, the average transition times reported in the previous section are representative of typical performance, whether or not the 5 second time limit is considered.

C. Performance Trends

The median transition times for both the data glove and MMC decreased with almost every subsequent trial until the sixth and final trial. For the final three trials, the median values for both the dataglove and MMC fell within 10% of their respective means. This was considered to be indicative of a performance plateau, and the final three trials were utilized to obtain the results reported here. Recall that each of five subjects attempted each transition type three times during each experimental task, and each of the three experimental trials involved tasks with both the dataglove and MMC. This resulted in 45 data points for each transition type (discounting unsuccessful attempts) for both the dataglove and the MMC. Recall that all trials used normalization parameters and twitch detection thresholds based on the EMG signals characterized before the first experiment was conducted. Additionally, placement of EMG electrodes was approximate and varied slightly between trials. These factors suggest that the MMC approach is robust to changes in EMG signals with respect to both time and placement.

D. Real-Time Control

Figure 3 shows the EMG inputs, hand state, and tendon excursions for a 55 second trial using MMC, where the user was sequentially prompted through the full range of hand postures. The figure demonstrates several important characteristics of MMC. First, the same EMG input can affect positional references for different tendons based on the current state of the hand. For instance, EMG input coming from the anterior forearm flexors (Electrode 1) generates references for T2 and T1 around the point and hook states, respectively (between t = 0 and t = 10 seconds). Second, a single EMG input may govern multiple tendons, such as Electrode 1 controlling both T1 and T3 between transitions from opposition to tip states around t = 33 seconds. Third, note that a high intensity co-contraction of flexor and extensor muscles results in a twitch event at t = 28 and t = 50 seconds.

Fig. 3.

EMG inputs, hand state, and tendon excursions as a user is prompted through a series of target postures.

V. Conclusion & Future Work

The multigrasp myoelectric controller presented in this paper enables the direct and proportional control of motion among several hand postures from a single EMG input (two bipolar electrodes). The cognitive effort required to operate a multigrasp hand while using the MMC is relatively low. This is because the MMC automates the low level control and coordination of the prosthesis. In this way the user need only convey high-level information regarding the direction and intensity of motion. The MMC also compares favorably to the native hand in terms of real time performance metrics; particularly with regard to transition completion rates. Subjects were able to familiarize themselves with the operation of the MMC within a few minutes, implying that the system provides an intuitive means of control. As only a single calibration was used over the one month trial period, and as electrode placement was approximate, this method appears to be robust with respect to spatial and temporal variation, although further testing will be required to consolidate this claim.

Future work will involve implementing the MMC on physical hardware and conducting experimental trials with amputees. Object detection and vibrotactile force feedback will be incorporated into the controller. Finally, variations of the finite state machine structure may be explored in order to further reduce transition times.