Engineering Platform and Experimental Protocol for Design and Evaluation of a Neurally-controlled Powered Transfemoral Prosthesis

Abstract

To enable intuitive operation of powered artificial legs, an interface between user and prosthesis that can recognize the user's movement intent is desired. A novel neural-machine interface (NMI) based on neuromuscular-mechanical fusion developed in our previous study has demonstrated a great potential to accurately identify the intended movement of transfemoral amputees. However, this interface has not yet been integrated with a powered prosthetic leg for true neural control. This study aimed to report (1) a flexible platform to implement and optimize neural control of powered lower limb prosthesis and (2) an experimental setup and protocol to evaluate neural prosthesis control on patients with lower limb amputations. First a platform based on a PC and a visual programming environment were developed to implement the prosthesis control algorithms, including NMI training algorithm, NMI online testing algorithm, and intrinsic control algorithm. To demonstrate the function of this platform, in this study the NMI based on neuromuscular-mechanical fusion was hierarchically integrated with intrinsic control of a prototypical transfemoral prosthesis. One patient with a unilateral transfemoral amputation was recruited to evaluate our implemented neural controller when performing activities, such as standing, level-ground walking, ramp ascent, and ramp descent continuously in the laboratory. A novel experimental setup and protocol were developed in order to test the new prosthesis control safely and efficiently. The presented proof-of-concept platform and experimental setup and protocol could aid the future development and application of neurally-controlled powered artificial legs.

Download video file ^{(37.2MB, mp4)}

Introduction

Powered lower limb prostheses have gained increasing attention in both commercial market^1,2 and research community^3-5. Compared to traditional passive prosthetic legs, motorized prosthetic joints have the advantage of allowing lower limb amputees to more efficiently perform activities that are difficult or impossible when wearing passive devices. However, currently, smooth and seamless activity transition (e.g., from level-ground walking to stair ascent) is still a challenging issue for powered prosthetic leg users. This difficulty is mainly due to the lack of a user-machine interface that can “read” the user’s movement intent and promptly adjust prosthesis control parameters in order to enable the users to seamlessly switch the activity mode.

To address these challenges, various approaches in designing user-machine interface have been explored. Wherein NMI based on electromyographic (EMG) signals has demonstrated a great potential to allow intuitive control of powered lower limb prostheses. Two recent studies^6,7 reported decoding the intended motion of the missing knee of transfemoral amputees by monitoring the EMG signals recorded from residual muscles during a seated position. Au et al.⁵ used EMG signals measured from residual shank muscles to identify two locomotion modes (level-ground walking and stair descent) of one transtibial amputee. Huang et al.⁸ proposed a phase-dependent EMG pattern recognition approach that can recognize seven activity modes with approximately 90% accuracy as demonstrated on two transfemoral amputees. To better improve the intent-recognition performance, a NMI based on neuromuscular-mechanical fusion was designed in our group⁹ and online evaluated on transfemoral amputees wearing passive prosthetic legs for intent recognition^10,11. This NMI can accurately identify the user’s intended activities and predict the activity transitions⁹, which was potentially useful for neural control of powered artificial legs.

The current question facing us is how to integrate our NMI into the prosthesis control system in order to enable intuitive prosthesis operation and ensure the user’s safety. Developing true neurally-controlled artificial legs requires a flexible platform in the laboratory for easy implementation and optimization of prosthesis control algorithms. Therefore, the objective of this study is to report a flexible engineering platform developed in our lab for testing and optimizing the prosthesis control algorithms. In addition, new experimental setup and protocol are presented for evaluating the neurally-controlled powered transfemoral prostheses on patients with lower limb amputations safely and efficiently. The platform and experimental design presented in this study could benefit the future development of true neurally-controlled, powered artificial legs.

Protocol

1. Platform for Implementation of Neural Control of Powered Transfemoral Prostheses

An engineering platform was developed in this study to implement and evaluate neural control of powered artificial legs. The hardware included a desktop PC with 2.8 GHz CPU and 4 GB RAM, a multi-functional data acquisition board with both analog-to-digital converters (ADCs) and digital-to-analog converters (DACs), a motor controller, digital I/Os, and a prototypical powered transfemoral prosthesis designed in our group¹². The analog sensor inputs were first digitized by the ADCs and streamed into the desktop PC for signal processing. The DAC was used for control output to drive the DC motor in prosthesis through a motor controller. Digital I/Os were used to enable/disable the motor controller. The powered prosthesis was tethered to the desktop PC and powered by a 24 V power supply.

The software was programmed in a development environment suitable for virtual instrumentation running on the desktop PC. The development environment was based on virtual instrumentation, which effectively combines both user-defined software and hardware to implement the customized platform. By using the structure of a graphical block diagram, different modular function nodes can be easily and efficiently implemented and updated. In order to demonstrate the function of platform for online control of powered artificial legs, preliminarily designed prosthesis control was implemented on this platform. The control system included a neural controller and an intrinsic controller. The neural controller consisted of our previous designed NMI based on neuromuscular-mechanical fusion, which recognized the user’s activity mode. The neural controller as a high-level controller was hierarchically connected with the intrinsic control for powered lower limb prosthesis control.

The architecture of control software on the platform is illustrated in Figure 1. The NMI contains two parts: offline training module and online testing module. The offline training module was designed to collect training data and build the classifiers in NMI. The collected multichannel surface EMG signals and mechanical measurements were first preprocessed and segmented into continuous sliding windows. In each window, features which characterize the signal patterns were extracted and then fused into one feature vector. The feature vector in each window was labeled with activity modes (classes) and phase index based on the performing activities of prosthesis user and the states of prosthesis during the training data collection. The labeled feature vectors were then used to build a phase-dependent pattern classifier, which contains multiple sub-classifiers correlated with individually phases. The created classifier was saved and transferred to online testing module for later online evaluation.

The online testing module was used to online recognize user’s movement intent and switch the activity modes in intrinsic controller. The multichannel neuromuscular and mechanical measurements were simultaneously streamed into online testing module and transformed into feature vectors. Then the feature vectors were fed into the phase-dependent classifier which was already built in offline training module. Based on the current phase in intrinsic controller, the corresponding sub-classifier was switched on and used to recognize the user’s intent. The classification output was further post-processed and sent to intrinsic controller to switch activity modes.

A finite-state machine (FSM) based impedance controller was implemented for the intrinsic control of powered artificial legs. The impedance controller generated desired torque output on the knee joints. The finite-state machine adjusted the joint impedance according to the current state of the performing activity. For locomotion activities (i.e. level-ground walking and ramp ascent/descent), the FSM consisted of five states corresponding to five gait phases: stance flexion (STF), stance extension (STE), pre-swing (PSW), swing flexion (SWF), and swing extension (SWE); for static standing, the FSM included two phases: weight bearing (WB) and non-weight bearing (NWB). Transitions between the states were triggered by the ground reaction force and knee joint position. The transition between activity modes was controlled by the output from the online testing module. For all three modules discussed above, graphical user interface (GUI) were built, which allowed experimenters in lab to easily adjust control parameters, monitor system performance, and conduct evaluation experiments.

2. Experimental Setup

1. Surface Electromyography (sEMG) Surface EMG signals from the thigh muscles of the amputee subject’s residual limb are recorded by a wireless EMG acquisition system. The targeted thigh muscles included the rectus femoris (RF), vastus laeralis (VL), vastus medialis (VM), biceps femoris long (BFL), sartorius (SAR), semitendinosus (SEM), and adductor magnus (ADM). It is noteworthy that precisely targeting the specific muscle is not necessary⁸ because the used pattern recognition algorithm in NMI searches activation patterns of multiple muscles to recognize the activity mode. As long as the measured neuromuscular control information for hip and knee control is sufficient, cross-talks among EMG recordings have little impact on the NMI performance.

   1. Ask the subject to perform hip flexion/extension, hip adduction/abduction, and attempt to flex/extend the amputated knee joint when he is in standing position.
   2. Approximate and determine the locations of electrode placements by muscle palpation and examination of EMG recordings.
   3. Embed the electrodes in a new designed EMG electrode-socket interface, as shown in Figure 2, for both the subject’s comfort and reliable electrode-skin contact. 
2. Mechanical Measurements from Powered Prosthesis The mechanical ground reaction forces and moments measured by a 6-DOF load cell from the prosthetic pylon are fused with EMG signals to recognize the subject’s movement intent. In addition, a potentiometer is instrumented on the knee joint to measure the knee joint angle and an encoder was connected to the DC motor to calculate knee angular velocity. These measurements are used as the feedback signals for intrinsic control.

   1. Mount a six-degree-freedom load cell on the prosthetic pylon.
   2. Align X-axis, Y-axis, and Z-axis of the load cell with the mediolateral direction, anteroposterior direction, and superoinferior direction of the prosthesis, respectively. 

3. Laboratory Environment Setup In order to evaluate the neural control of powered artificial legs on transfemoral amputees, an obstacle course was built in the laboratory, as shown in Figure 3. The testing environment included a 5-m straight walkway, a 4-m long ramp with an inclination angle of 8°, and a level platform with the ramp securely attached. Along the ramp, hand railings were installed to enhance the safety of the subject. In addition, a ceiling rail system with a fall-arrest harness was provided to protect the subject from falling during the experiment.

3. Experimental Protocol

This study was conducted with the approval of Institutional Review Board (IRB) at the University of Rhode Island and with informed consent of the recruited subject. One male unilateral transfemoral amputee (cause of amputation: trauma; age: 57 years; duration of amputation: 32 years) was recruited in this study. The ratio between the length of the residual limb (measured from the ischial tuberosity to the distal end of the residual limb) to the length of the non-impaired side (measured from the ischial tuberosity to the femoral epicondyle) was 51%. The subject wears a microprocessor-controlled prosthetic knee through a suction suspension socket in his daily life. Prior to the experiment in this study, this subject received several training sessions led by a physical therapist in order to let the subject to adapt to the powered device and calibrate the desired impedance in each activity mode.

1. Subject Preparation

   1. Measure the subject’s weight, height, and record his gender and age.
   2. Ask the subject to put on his own shorts in a private preparation room.
   3. Put a size-fitted fall-arrest harness on the subject and attach it to the ceiling rail system. 
2. Preparation for EMG Recording

   1. Select seven fully charged wireless EMG sensors and turn them on.
   2. Place the EMG sensors into the customized suction socket at prepared locations. Write down the order number of the sensors and associate them with EMG locations.
   3. Clean the skin of subject’s residual limb with isopropyl alcohol pads.
   4. Assist the subject in donning the suction socket and verify that the socket is firmly attached to subject’s residual limb.
   5. Turn on the real-time EMG analog data streaming software.
   6. Ask the subject to perform hip flexion/extension, hip adduction/abduction, and knee flexion/extension and examine EMG signals to verify EMG electrode contact and data transmission.
3. Alignment and Initial Calibration of Powered Prosthetic Leg

   1. Instruct the subject to remain in a standing position while holding an assistive walker.
   2. Attach the powered prosthesis to the suction socket with a pyramid adaptor. Adjust a set of rotation screws on the adaptor until the position of the prosthesis is geometrically aligned with the socket. This procedure was performed by a prosthetist.
   3. Ask the subject to lift the prosthesis off the ground and calibrate the load cell on the prosthetic pylon.
   4. Instruct the subject to practice walking on different terrains (e.g., level-ground, ramp ascent, and ramp descent) when wearing the powered prosthetic leg. This procedure continues until the subject feels confident in walking with the powered device and yields consistent gait pattern in each activity performance. 
4. Training Data Collection for Training the Classifiers in NMI

   1. Instruct the subject to stand on the starting location of a predefined walking path, as shown in Figure 3.
   2. Turn on the powered prosthesis and load the parameters into the intrinsic controller.
   3. Run a training data collection computer program and set the intrinsic control to standing mode by clicking the “Standing” button on the graphic user interface (GUI).
   4. Begin data collection by clicking the “Start Recording” button on GUI. Instruct the subject to stay in standing position for 5 sec.
   5. Instruct the subject to walk over level ground at his/her self-selected comfortable walking speed; at the same time, click the “Walking” button on the GUI before toe-off of the leading leg of the subject and set the intrinsic control to level-ground walking mode.
   6. When the subject was approaching to the edge of ramp ascent, click the “Ramp Ascent” button on the GUI before the toe-off of the prosthetic leg stepping on the ramp and switch the intrinsic control to ramp ascent mode. For safety, allow the subject to use a hand railing when walking on a ramp.
   7. When the subject comes to the end of the ramp, click the “Walking” button again before the heel strike of prosthetic leg stepping onto the level platform and switch the prosthesis intrinsic control to level-ground walking mode.
   8. At the end of the walking path, instruct the subject to stop and remain in standing position. At the same time, click the “Standing” button before the double stance phase and switch the intrinsic control back to standing mode.
   9. After around 5 seconds, terminate data collection by clicking “Stop” button. Label collected data as “training data set 1”.
   10. Repeat the procedure 3.4.4-3.4.9 when the subject walks in a reverse route back to the starting location; the only difference is switching the intrinsic control to ramp descent mode when the subject walks on the down ramp.
   11. Repeat 3.4.4-3.4.10 until ten complete training data sets are collected. Examine the signal quality of collected training data set.
   12. Allow the subject to have a rest period after the data collection session.
   13. Train the pattern recognition classifiers in NMI via offline training module (Figure 1). Use the collected EMG and mechanical signals, the activity modes (classes) labeled during the training procedure, and detected phases to build the phase-dependent pattern classifiers. Save parameters of the classifiers automatically for later online testing session.
5. Online Testing of Neural Control of Powered Transfemoral Prosthesis

   1. Instruct the subject to stand at the starting point of the walking path.
   2. Turn on the powered prosthesis. Load the trained classifier to online testing module and the parameters to the intrinsic controller.
   3. Instruct the subject to begin the testing trials in a standing position, then continuously transition to level-ground walking, ramp walking, level-ground walking, and finally stop and finish this trial at the end of the walking path. Instruct the subject to perform each activity at a comfortable pace. Allow rest periods between trials to avoid fatigue.
   4. During each testing trial, display the activity modes of the prosthesis and knee joint angle readings on a TV monitor. Save all of the measurements and control outputs for later evaluation purpose.
   5. Repeat steps 3.5.1-3.5.4 until ten complete testing trials are finished. 

Representative Results

Figure 4a shows seven channels of surface EMG signals measured from the thigh muscles of the subject’s residual limb when he performed hip flexion/extension, as described in Protocol 3.2.6. Figure 4b shows six gait cycles of EMG signals recorded when the subject walked on a level-ground walking path, during Protocol 3.3.4. From this figure, it can be seen that the new designed EMG electrode-socket interface can provide good quality of surface EMG signal measurements.

Figure 5 demonstrates control modes, detected phases, and resulted knee joint angle of neurally-controlled powered transfemoral prosthesis in one representative online testing trial. The subject was asked to begin in a standing position, transition to level-ground walking, ramp ascent, level-ground walking, and then stop at the end of the walking path. The subject then returned to the original starting point along the reverse route. As shown in Figure 5, with the neural control, the recruited subject was able to smoothly switch the powered transfemoral prosthesis control mode based on his intended activity modes. The red dash line indicated our defined critical timing of each activity mode transition. For transition from level-ground walking to ramp ascent/descent and from standing to walking, the critical timing was beginning of the swing phase (i.e., toe off) in the transitional periods; for transition from ramp ascent/descent to level-ground walking and walking to standing, the critical timing was the beginning of weight acceptance (i.e., heel contact) on the level ground. About 18 seconds into this representative trial, the prosthesis incorrectly switched to ramp ascent mode when the subject walked on level-ground due to the erroneous recognition of user’s intent by the NMI. These errors did not elicit significant change in walking kinematics of the subject and were not perceived by the subject. However, we did observe some errors which disturbed the subject’s gait stability in some testing trials. But none of these errors caused the subject to fall.

Figure 1. The architecture of control software on the testing platform for evaluation of online neural control of a prototypical powered transfemoral prosthesis. Please click here to view a larger version of this figure.

Figure 2. a) Frontal and sagittal view of the subject wearing the powered transfemoral prosthesis with the EMG and mechanical measurement setup; b) Data collection cart with the desktop PC, wireless EMG base station, multifunctional DAQ board, and power supply. Please click here to view a larger version of this figure.

Figure 3. Experimental obstacle course built in the laboratory environment.

Figure 4. Raw EMG signals recorded from the thigh muscles of subject’s residual limb, when the subject (a) performed hip flexion/extension; (b) walked on a level-ground walking path.

Figure 5. Example of control modes, detected phases, and resulting powered knee joint angle of neurally-controlled powered transfemoral prosthesis in one representative online testing trial. The red dash line indicates the defined critical timing of each activity mode transition.

Discussion

An engineering platform was developed in this study to easily implement, optimize, and develop true neural control of powered prostheses. The whole platform was programmed in a virtual instrumentation based development environment and implemented on a desktop PC. The control software was composed of several independent and interchangeable modules, in each of which a specific functionality was executed (i.e. NMI intent recognition, and intrinsic control). The advantage of this modular design is that each function block can be easily debugged, modified, and updated. In addition, adding or deleting functions or changing the connections between modules can be easily done in this computer program. Therefore, the platform can be easily applied to implement other NMI based on different neural control information or based on different user intent recognition techniques. For example, when an EEG-based neural-machine interface is developed, it can be easily implemented on this platform by modifying the data acquisition module and intent recognition module; when a neural-machine interface based on a new intent decoding method is developed, it can be applied by simply replacing the intent recognition module. However, it should be noted that the software of this engineering platform was designed based on a hierarchical structure which connected a high-level intent recognition module with a low-level prosthesis controller. Therefore, only the neural-machine interface that can decode and output the user’s movement intent can be easily applied on this platform. Additionally, note that this platform is only used to prove the concept. Once the neural control of powered artificial legs has been finalized, essentially the controllers should be implemented on a powerful embedded computer system and integrated within the prosthesis in order to make the artificial legs practical and portable for use on daily bases.

An experimental setup was designed to test the neural controller for powered artificial legs on patients with transfemoral amputations. Two important aspects in our setup are highlighted. First, ensuring safety of the recruited subject is necessary. This is because misclassifications in NMI (as an example shown in Figure 5) or errors in intrinsic control may occur in the initially designed prosthesis control, which could disturb the prosthesis user’s walking stability (observed in this study although not all the NMI errors disturb walking stability), lead to falls, and threat their safety. Hence, in our setup hand railings were installed for uneven terrains. In addition, a ceiling rail system with a fall-arrest harness was built, which covered the entire experimental area in the laboratory and was applied to maximally protect the subject’s safety. Second, it is critical to develop a comfort and reliable EMG electrode-socket interface. Inserting sensors into prosthetic sockets may build high pressure or friction against certain regions of residual limbs, which leads to discomfort in walking and poor gait pattern in the subject. Approaches that mitigate the pressure/friction may, however, cause loose contact between electrodes and the residual limb, resulting in noisy EMG recordings. In this study, we presented a new electrode-socket interface based on wireless EMG electrodes, which provided high quality in EMG signals for accurate intent recognition (as demonstrated in Figure 4), tight socket suspension, and good user comfort. Although some cross-talks were observed in a few channels (i.e. Channel 3 and 6 in Figure 4b), they have little impact on the performance of NMI. It is because the pattern recognition algorithm designed in NMI searches activation patterns of multiple muscles to recognize the activity mode. This setup may also benefit other research or clinical groups, who plan to investigate the muscle property or function in residual limbs of lower limb amputees.

In order to efficiently evaluate the neurally-controlled artificial legs, experimental trials were designed to include all the studied activity modes and mode transitions. It is noteworthy that the gait and balance training of lower limb amputees in using powered prostheses is necessary prior to the presented experiment in order to make the NMI accurate in recognizing the user’s activity mode. This is because pattern recognition is used in our NMI, which recognizes different activity mode by searching the EMG and mechanical signals pattern that is consistent for one activity mode but different from the other modes. The training sessions ensure users to adapt to the powered devices that redefine their dynamics in walking and produce consistent gait pattern when performing the same activity. Additionally, for the same purpose, the subject should be given time at the beginning of the experiment to acclimate to the powered prosthesis and achieve a smooth and consistent walking pattern.

Finally, we want to emphasize that the flexible testing platform, experimental setup, and experimental protocol presented in this study were useful for laboratory development and evaluation of neural control for powered artificial legs. In order to make finalized neurally-controlled artificial legs practical for daily use, development of embedded engineering platform and user-friendly calibration procedure and evaluation in realist environments are necessary in the future. Additionally, the neural control presented in this study is preliminary and used to demonstrate the function of our testing platform and experimental design only; it is not the final control for powered devices as we did observe errors in NMI that disrupt the walking pattern of lower limb amputees, which should be eliminated. Our reported platform and evaluation setup and protocol provided convenient tools to further optimize the neural controller and intrinsic control and develop a true bionic lower limb prosthesis that can be operated by the users easily, reliably, and intuitively.

Disclosures

No conflicts of interest declared.