Patient training for functional use of pattern recognition–controlled prostheses

Abstract

Pattern recognition control systems have the potential to provide better, more reliable myoelectric prosthesis control for individuals with an upper-limb amputation. However, proper patient training is essential. We begin user training by teaching the concepts of pattern recognition control and progress to teaching how to control, use, and maintain prostheses with one or many degrees of freedom. Here we describe the training stages, with relevant case studies, and highlight several tools that can be used throughout the training process, including prosthesis-guided training (PGT)—a self-initiated, simple method of recalibrating a pattern recognition–controlled prosthesis. PGT may lengthen functional use times, potentially increasing prosthesis wear time. Using this training approach, we anticipate advancing pattern recognition control from the laboratory to the home environment and finally realizing the full potential of these control systems.

Introduction

The ultimate goal of pattern recognition control systems is to provide individuals with an upper-limb amputation with better, more reliable control of their myoelectric prostheses. Pattern recognition allows individuals to control their prosthesis using physiologically appropriate muscle contractions and eliminates the need for complicated mode switching. Finding isolated myoelectric control sites is largely unnecessary; pattern recognition allows individuals to use a myoelectric prosthesis despite having control sites with poor myoelectric signal separation or sites that pose a challenge for prosthetic fitting. Increasing the utility of prostheses with one or many degrees of freedom through pattern recognition control may lead to increased wear time in the home and community.

This paper outlines a clinical protocol for training individuals to use and maintain pattern recognition control of upper extremity myoelectric prostheses. The protocol has been developed through our research and clinical experience at the Rehabilitation Institute of Chicago.^1–5 Training is divided into four stages (Figure 1):

1. Conceptual Training: Teaching pattern recognition concepts and establishing which attempted movements will be used to control each degree of freedom of the prosthesis.

2. Control Training: Providing guidance while the user learns to control the motions of a prosthesis using pattern recognition.

3. Functional Use Training: Teaching prosthesis use and bimanual function during activities of daily living.

4. Prosthesis Recalibration Training: Teaching the individual how to maintain performance of their pattern recognition–controlled prosthesis during everyday usage.

Figure 1.

Overview of the four training stages and the major goals within each stage.

We identify goals and tools for each training stage and highlight the clinical application of each stage through case studies of individuals with an amputation at the transradial, transhumeral, or shoulder disarticulation level. Some case studies involve individuals who have undergone targeted muscle reinnervation (TMR) surgery^2,6,7 and some examine individuals who have not undergone TMR. This surgery is not necessary for successful implementation of pattern recognition but it can provide enhanced, intuitive control when coupled. Training guidelines outlined in this paper can be followed for TMR and non-TMR patients.

Conceptual Training Stage

Conceptual training involves introducing and developing an understanding of myoelectric pattern recognition control.⁵ This phase does not require use of a prosthesis and so can be initiated prior to and during initial socket or liner fabrications. This stage should be tailored to individuals; their rate of progression will depend on whether they previously used a myoelectric prosthesis, the ease with which they understand the concepts, and the number of degrees of freedom to be controlled.

We encourage individuals to actively participate in training, including having discussions about their control with the clinician. The process of selecting a shared vocabulary, such as signal, degree of freedom, or palm up, ensures clear communication between the individual and the clinician. Discussions should also include whether or not the phantom limb will be useful during pattern recognition training. Users should be initially instructed to move their phantom limb in the desired way, even if it feels immobile. If individuals experience phantom limb pain or if the phantom sensation distracts from learning, we ask users to instead focus on mirroring the desired movement with the intact limb.

Using the established vocabulary, training begins with verbal explanations of important concepts; specifically, that in pattern recognition control the patterns of muscle activity from all electrode sites are used to control each movement of the prosthesis. Previous users of myoelectric devices should understand that specific electrode locations no longer correspond to specific movements as is the case in conventional myoelectric control. Users should also understand that, for good pattern recognition control, each attempted movement should be performed the same way each time. Individuals are instructed to perform consistent muscle contractions at a moderate level of effort. Strong muscle contractions are not required to obtain good movement differentiation and should be discouraged because muscles will fatigue quickly. Since current pattern recognition systems only allow for sequential movements, individuals need to learn to perform one movement at a time. It will take repetition and time for users to understand and use these concepts; since this is a necessary part of achieving good control, these concepts should be reviewed at all subsequent training stages.

Goals of Conceptual Training

1. To create a shared vocabulary.

2. For previous myoelectric users, to identify the differences between conventional direct control and pattern recognition control.

3. To establish and practice the set of muscle contractions that will ultimately be used to control degrees of freedom in the pattern recognition–controlled prosthesis.

4. To understand pattern recognition concepts including the need to perform consistent muscle contractions related to movements available in the prosthesis, to use a moderate level of effort, and to focus on moving degrees of freedom sequentially.

Tools for Conceptual Training

We employ several tools to further the understanding of pattern recognition control including a myoelectric signal viewer, screen-guided training, a virtual reality environment, and a home exercise program. These tools are available at the start of conceptual training and remain valuable resources throughout the entire training process. The tools that are used and the amount of time spent with each tool will depend on the individual’s needs, their learning style, and the time available prior to their being fitted with a prosthesis. Based on our clinical experience, we encourage exploring the use of each tool at least once, and then focusing on the tools that best aid each individual’s understanding and use of pattern recognition control.

Myoelectric Signal Viewer

Once electrode sites are selected, we use a multi-channel myoelectric signal viewer to illustrate to the individual how different movements produce different patterns of muscle activity (Figure 2A). The signal viewer can be used to emphasize the importance of performing muscle contractions the same way each time with a moderate level of effort; varying levels of effort will produce varying patterns of muscle activity. It is important to explain that we are not looking for signal independence between two or more electrode channels, rather that the set of signal patterns differ for each movement, and that the signal pattern for each movement is repeatable. This tool is highly visual, yet somewhat abstract, and may not be suitable or useful for all clinicians and users.

Figure 2.

Conceptual training tools: A) Screen shot from multi-channel myoelectric signal viewer showing the different muscle activation patterns for wrist flexion and wrist extension. Six myoelectric channels are used in this example. B) SGT image used to instruct users when to perform specific muscle contractions for calibration of the pattern recognition system. C) An individual with a shoulder disarticulation post-TMR surgery using virtual reality environment to practice pattern recognition control. D) An individual with a transradial amputation remotely operating a prosthesis.

Screen-Guided Training (SGT)

Pattern recognition control requires a set of myoelectric signals, corresponding to each possible movement of the user’s prosthesis, to be recorded and used to calibrate the control system. SGT is one method for acquiring these myoelectric signals. Individuals are instructed to perform muscle contractions in synchrony with a sequence of images displayed on a monitor^2,8,9 (Figure 2B). Individuals need to be connected to a computer to use SGT, but it is not always necessary to have an actual prosthesis and socket. By means of a check socket, a diagnostic liner, or temporary electrode contacts, users can use SGT as a tool to calibrate and practice pattern recognition control at different stages in prosthetic fitting.

Virtual Reality Environment

Once the pattern recognition system is calibrated using the individual’s muscle signals, he or she can use pattern recognition control with a virtual prosthesis^{2, 10–12} (Figure 2C). Since this may be the individual’s first experience with real-time control, we recommend beginning with the movements that are easiest to perform; hand open and close for individuals with a transradial amputation, and elbow up and down for individuals with a transhumeral amputation. Individuals should practice moving the virtual prosthesis, including starting and stopping movements, both following the commands of the clinician and under their own guidance. Frequent recalibration of the pattern recognition system is necessary during initial training sessions. Physiological changes such as altered skin conduction or alterations in attempted movements that occur as the individual adapts to the training process will cause degradation in real-time control. We explain to individuals that the need for frequent recalibration is expected, although it may become less frequent as they gain experience in using pattern recognition control.

Once the individual can control easy movements, we gradually add degrees of freedom up to the capability of their definitive prosthesis. Since these movements may be outside of the individual’s experience, fatigue will be common as they practice new muscle activation patterns. Frequent rest breaks and recalibration are necessary when real-time control degrades.

Subsequent training sessions and use of the virtual reality environment can include practice of more pattern recognition concepts. If any unintended movements of the virtual prosthesis occur, the individual can be coached in making consistent muscle contractions with a moderate level of effort and fully resting their residual limb when they want the prosthesis to stop moving. When control degrades in the virtual environment, remind the individual that performing stronger muscle contractions will not lead to better control. Often it is helpful to ask the individual how the contractions felt during the calibration session—performing the contraction with a similar feeling and intensity may improve real-time control.

Virtual environment assessment tools such as the Motion Test^1,2 and the Target Achievement Control Test³ may be beneficial for additional practice of pattern recognition concepts. These virtual tests prompt the individual to position the virtual prosthesis in a variety of ways, providing feedback on the speed and efficiency of their control that can further motivate practice of pattern recognition control.

Remote Operation of a Prosthesis

If a prosthesis is available, individuals can use pattern recognition to control it remotely. The prosthesis can be mounted on a table and the myoelectric signals collected either through a test socket or temporary electrode contacts (Figure 2D). This tool, much like the virtual reality environment, can provide further practice of pattern recognition concepts and in addition, the individual can learn the different prosthetic movements including the grasping pattern of the prosthetic hand (i.e. the point of contact for the thumb and the index finger).

Home Exercise Program

Patients can be given a home exercise program that focuses on practicing the muscle contractions necessary for pattern recognition control of their prosthesis. Home programs can help strengthen muscles and so reduce the muscle fatigue associated with learning new muscle activation patterns. Photographs of the movements to be performed (Figure 3) and guidelines as to how often to perform these movements can be sent home with the individual at the end of the first session. The home program can serve as a good tool for communication between clinician and patient during the period before prosthetic fitting or during wound healing.

Figure 3.

Examples from a home program document showing exercises for an individual with a transradial amputation. Pictures correspond to the actions of the phantom limb and movement descriptions use the established vocabulary.

Case Study

Deciding on attempted movements to control the prosthesis

A 61-year-old male myoelectric prosthesis user with a unilateral transradial amputation needed to learn new motor commands for a pattern recognition–controlled prosthesis. With conventional direct control, he had used wrist flex and extend to close and open respectively, whereas for pattern recognition control of the hand, he needed to use the physiologically appropriate muscle contractions generated by opening and closing his phantom hand. We began training by using a myoelectric signal viewer to practice these contractions. We discussed whether or not his phantom fingers were spread apart while he opened his hand, and whether his thumb was enclosed by his fingers while closing his hand. We progressed to adding wrist rotation, and he described leading each rotation with his phantom thumb. When a verbal description of how he accomplished an attempted movement was unclear, he demonstrated it with his intact limb. Since pattern recognition control relies on making consistent and repeatable muscle contractions, this information was recorded and used as a reminder during subsequent visits. Successful training in this case relied on good communication between clinician and patient

Control Training Stage

Control training involves teaching the individual to control an actual prosthesis using pattern recognition. This stage can be initiated once a socket with embedded electrodes is fabricated and the prosthesis is available. The weight of the prosthesis, proximal postural effects, and prosthesis position affect conventional myoelectric control and will also affect pattern recognition control.¹³

Learning to control a physical prosthesis can be more difficult than learning to control a virtual prosthesis. We recommend using the same approach of beginning with the easiest movements to perform. If the easiest movements are hand open and close, prompt the user to open the hand all or part of the way. Explore different workspaces that require different residual limb positioning, including moving the limb through space, with the goal of achieving the desired prosthesis movement without unintended movements.

If the prosthesis has multiple degrees of freedom, we recommend adding new movements only when the individual is confident controlling each previous movement. Adding too many movements too early will quickly degrade control. Prompt the individual to perform each movement of the prosthesis through its full range of motion, with and without mirroring with the intact limb, and in different sequential orders. Transradial users should perform movements both with their forearm unsupported and supported (e.g., resting on a table). It is important to note when unintended movements occur and to practice the desired movements under those conditions, recalibrating the prosthesis when necessary.

Frequent reminders of pattern recognition concepts (e.g., to perform consistent contractions at a moderate level of effort) are necessary. When control degrades while using the prosthesis, individuals often resort to performing stronger muscle contractions in hopes of regaining control. Trying too hard or changing the way muscle contractions are performed are the two most common causes of poor control. Fatigue can also contribute to poor control, especially because the individual now has to contend with the weight of the prosthesis on the residual limb; allowing sufficient rest breaks is important during this training phase.

Goals of Control Training

1. To control all degrees of freedom of the prosthesis.

2. To be able to perform consistent muscle contractions, use a moderate level of effort, and perform only one movement at a time while controlling the prosthesis.

3. To recalibrate the prosthesis when prompted by the clinician.

Tools for Control Training

Prosthesis-Guided Training (PGT)

PGT is an easy, intuitive method of collecting the myoelectric signals necessary to recalibrate a pattern recognition–controlled prosthesis.¹⁴ Individuals can self-initiate recalibration without needing to connect to an external computer and/or monitor. During PGT, the prosthesis moves through a sequence of movements. The individual follows along by producing the corresponding muscle contractions and relaxing each time the prosthesis pauses between movements (Figure 4). The prosthesis goes through the same sequence and timing of movements during each recalibration. This likely helps the individual to perform the consistent, repeatable muscle contractions necessary for effective recalibration. Since myoelectric signals are collected while the prosthesis is moving with the residual limb in any position, the recalibration environment is similar to that encountered during real-time use.

Figure 4.

Individuals with a transhumeral amputation performing prosthesis-guided training.

Therapist Instruction

Prompting individuals to perform control exercises can provide repetitive practice and identify prosthetic movements that are difficult to perform. Starting with each individual degree of freedom, the clinician should prompt the individual both to move through the full range of motion and to stop half way through. Prompts can also request slow, fast, fine, or gross movements. If the prosthesis has multiple degrees of freedom, the prompted movements should alternate between them. For example, for a powered hand and elbow, the sequence of movements may be: bend the elbow all the way up, straighten the elbow, bend the elbow half way up, open the hand, close the hand, open the hand half way, bend the elbow, straighten the elbow, etc.

Case Study

Initial Introduction to PGT

A 33-year-old male with a unilateral transhumeral amputation who had undergone TMR surgery was introduced to PGT once his socket had been fabricated. During the first few times he recalibrated his prosthesis using PGT he realized that the more attention he paid to the process, the better the prosthesis worked for him. Through coaching, he learned to use his intact limb to mirror the movements of both the prosthesis and his phantom limb during PGT (Figure 4). He stated that incorporating both limbs allowed him to focus more on making consistent muscle contractions with his phantom limb.

Functional Use Training Stage

Functional use training involves teaching the individual to use the prosthesis in a functional manner, which includes handling objects and performing bimanual activities of daily living (5). This stage can be initiated once the individual has demonstrated the ability to control all degrees of freedom of the prosthesis. Functional use and control training can occur within the same training session, switching between them as necessary. The addition of functional, bimanual, and cognitive demands will all affect the individual’s ability to control the prosthesis.

For functional prosthesis control and use, we introduce common objects for grasping and change their orientations to provide the user with practice at prepositioning their device. If the pattern recognition–controlled prosthesis is capable of more movements than the individual’s previous prosthesis, it is useful to guide the individual to appropriately preposition the device as they use the new functions instead of allowing them to continue any postural accommodations used with the previous device (Figure 5).

Figure 5.

Individual with a transradial amputation being coached away from using postural accommodations and to use the prosthesis wrist flexion.

The next step is to incorporate bimanual tasks (Figure 6). Bimanual function is essential and will initially require increased instruction because the focus up to this point has been on the prosthesis with the intact limb often taking a more passive role. Simultaneous action involving both limbs such as picking up and carrying a lunch tray, folding laundry, and using a tape measure can provide bimanual practice. Tasks such as hanging clothes, taking money out of a wallet, using scissors, and making a sandwich require the individual to alternate between use of the intact limb and the prosthesis. We encourage individuals to minimize the amount of visual attention paid to the prosthetic terminal device.

Figure 6.

Individuals performing bimanual tasks and activities of daily living.

Once the individual reports satisfaction with prosthesis performance and can demonstrate basic bimanual skills, the cognitive demands of functional tasks can be increased. Activities including preparing a meal, packing a suitcase, assembling a bookshelf, or sewing on a button require more organization and planning. Verbal cues should be given for prepositioning, and recalibration may be needed if unusual device positions affect control.

As the complexity of the tasks increases, pattern recognition control may decline. It is important to determine whether recalibration of the prosthesis is necessary as not all control degradation requires recalibration. Muscle fatigue may best be remedied by rest, while environmental changes (e.g., sweat buildup) may require recalibration. Inconsistency in muscle contraction patterns or intensity is other potential causes of poor control. If reviewing these concepts does not improve control during functional use, ask the individual to control the device without handling an object. If the user is able to move the prosthesis correctly through the range of motion for each movement then recalibration may not be necessary and bimanual practice can resume. If not, then recalibration is indicated. Help the individual to judge the reason for the degraded control and determine the best solution. With PGT, individuals can seamlessly transition from using the prosthesis to recalibration and back to using the device in approximately one minute. Recalibration using PGT should gradually occur less often through clinician-prompting and move towards self-initiation by the individual.

Goals of Functional Use Training

1. To properly preposition the prosthesis when handling objects.

2. To perform bimanual tasks and activities of daily living using the prosthesis.

Tools for Functional Use Training

Videotaping Tasks

Allowing individuals to watch videos of themselves using the prosthesis is instructive in demonstrating progress. Watching video of others who have mastered pattern recognition control of a similar device can demonstrate the potential of pattern recognition control.

Southampton Hand Assessment Procedure (SHAP)

The SHAP is a clinically validated evaluation of hand function consisting of self-timed tasks involving abstract objects and simulated activities of daily living.¹⁵ This tool can be useful for assessing prepositioning skills and can be used to measure progress.

Assessment of Capacity for Myoelectric Control (ACMC)

The ACMC measures an individual’s ability to control the myoelectric prosthesis in a functional context.^16,17 This clinically validated tool can be useful for assessing bimanual function and can be used to measure progress.

Case Studies

Learning to use a new prosthesis function

A 36-year-old male with a unilateral transradial amputation was learning to use a prosthesis that provided powered wrist flexion and extension. He was instructed to remove bills from a wallet. In his first attempt, he held the wallet with the prosthesis and had difficulty removing the bills because of its tight grip. He did not use wrist flexion. We pointed out that he could use the wrist flexion feature to better complete the task. Through verbal prepositioning cues, he then supinated and flexed the wrist. In this new position he was able to see into the wallet better and to loosen his grip and remove the bills without dropping the wallet. After practicing the task several times, he was then encouraged to hold the wallet in his intact hand while removing the bills with the prosthesis, now incorporating more wrist flexion in order to reach into the wallet.

Learning to complete a functional task successfully

A 31-year-old female with a unilateral shoulder disarticulation, and who had undergone TMR surgery, was learning to sort laundry using her pattern recognition–controlled prosthesis. Initially she had difficulty letting go of each item after extending her elbow down to the height of the laundry basket. With the elbow in this extended position, we prompted her to open and close the prosthetic hand without trying to hold or release an item. She was able to do this successfully but still had difficulty when we returned to the task. We asked her to verbalize what she was trying to do as she did it. Through this exercise we realized that she was performing simultaneous elbow extension and hand open muscle contractions. We reminded her that the current pattern recognition system only allows one movement to be performed at a time. Once she focused on making sequential muscle contractions to first extend the elbow and then open the hand she was able to successfully pick up items and release them in the laundry basket.

Prosthesis Recalibration Training Stage

Prosthesis recalibration training involves teaching the individual how to retain functional use of their pattern recognition–controlled prosthesis if control degrades during daily use. This stage can be initiated once the individual has demonstrated good functional control of their prosthesis.

Individuals can encounter several different types of issues in their home and community that can cause their prosthesis control to degrade. Differences in prosthesis donning, environmental changes, muscle fatigue, and electrode problems can all occur during day-to-day use. Therefore, before leaving the clinic, they should practice donning the prosthesis and evaluating control. Calibration of their prosthesis using PGT may be necessary if their socket is donned slightly differently so that the electrode positions are different from last use¹⁸ or if the user changes the movements used for control. Prompting the individual to move the prosthesis throughout its full range of motion after donning will allow him or her to test their control and determine if recalibration is necessary. We encourage users to perform a similar type of control check any time they encounter difficulty controlling their prosthesis.

Goals of Prosthesis Recalibration Training

1. To self-initiate recalibration using PGT at appropriate times.

2. Successful restoration of control after recalibration.

Tools for Prosthesis Recalibration Training

PGT for Regaining Prosthesis Control

PGT was developed so that individuals can take an active role in maintaining acceptable control of their prosthesis. PGT uses the current patterns of myoelectric activity to recalibrate the pattern recognition system. If myoelectric signal patterns are altered because the residual limb is sweating or if muscle activity is altered due to fatigue, these characteristics will be used for recalibration and allow functional use under these conditions.

Recalibration through PGT also may provide individuals with longer functional use times before the prosthesis must be taken off and taken back to the clinic for repairs. Pattern recognition control often uses more electrodes than conventional myoelectric control and since the myoelectric signals from all electrodes contribute to control of each movement, failure or breakdown of any electrode could, in theory, drastically reduce overall control. Recalibration through PGT allows the faulty electrode signal to be part of the signal set, potentially restoring control that, although perhaps not as good, may be adequate for continued use of the prosthesis, for example, while awaiting a clinic appointment.

Case Studies

Reactions to Using PGT for Prosthesis Recalibration

Five individuals who had undergone TMR surgery, used a myoelectric prosthesis, and had experience with pattern recognition systems including recalibration using SGT and PGT were surveyed regarding their opinions of PGT.¹⁴ Averaged responses indicated a willingness to “refresh” or “recalibrate” their prosthesis up 3.2 (SD 1.7) times per day or no more than about every 2.4 (1.6) hours. While performing PGT, individuals occasionally perform the necessary muscle contractions incorrectly or inconsistently. If this occurs, they most likely will not have good real-time control and will need to recalibrate again. Surveyed individuals stated they would be willing to perform PGT up to 3.2 (1.5) times in a row in an attempt to restore good control, instead of removing the prosthesis.

Extending Prosthesis Wear Time through PGT

Five individuals who had undergone TMR surgery had an opportunity to use a 4 degree of freedom pattern recognition–controlled prosthesis while real-world electrode problems (e.g. broken wires and noisy electrodes) were simulated to accelerate control degradation.¹⁹ They were asked to continue performing a clothespin relocation task and to self-initiate recalibration using PGT when they believed their performance had declined. The task ended when they permanently lost functional control and indicated they wanted to remove their prosthesis. Given the opportunity to recalibrate their prostheses, individuals were able to place more clothespins and use their prosthesis for longer times, overcoming an average of two (out of eight) non-functioning electrodes before losing control of their devices (Figure 7). Without the opportunity to recalibrate their prosthesis using PGT, these individuals would have had no choice but to take off their prosthesis after the first electrode signal was corrupted.

Figure 7.

Number of clothespins placed vs. time for individuals with a transhumeral amputation (T5) and a shoulder disarticulation amputation (S1). The effects of the simulated broken electrodes were cumulative in nature. PGT allowed both individuals to maintain prosthesis control even after multiple electrode faults.

Conclusions

For individuals with an upper-limb amputation, learning to perform activities of daily living with a pattern recognition–controlled prosthesis takes time and practice because it can differ from the conventional style of control they may be accustomed to. This paper provides a clinical protocol to help achieve this goal. The guidelines, created through the integration of clinical training and technology developed at the Rehabilitation Institute of Chicago, can be used with a wide range of upper extremity prosthesis users. Individuals with different levels of amputation, who have or have not undergone TMR surgery, and those who use a prosthesis with one or several degree(s) of freedom all have the potential to successfully use pattern recognition control.

The option to self-initiate recalibration through PGT, and the ability to know when to do so, may provide individuals with a longer time frame for functional prosthesis use. This feature, together with the ability of pattern recognition to use intuitive muscle contractions for prosthesis control, has the potential to increase prosthesis wear time, improve function, and reduce device rejection. We anticipate advancing pattern recognition control from the laboratory to the home environment and further refining our training protocol to further improve user function.