Preliminary Results for an Adaptive Pattern Recognition System for Novel Users Using a Powered Lower Limb Prosthesis

Abstract

Powered prosthetic legs are capable of improving the gait of lower limb amputees. Pattern recognition systems for these devices allow amputees to transition between different locomotion modes in a way that is seamless and transparent to the user. However, the potential of these systems is diminished because they require large amounts of training data that is burdensome to collect. To reduce the effort required to acquire these data, we developed an adaptive pattern recognition system that automatically learns from subject-specific data as the user is ambulating. We tested our proposed system with two able-bodied subjects ambulating with a powered knee and ankle prosthesis. Each subject initially ambulated with a pattern recognition system that was not trained with any data from that subject (making each subject a novel user). Initially, the pattern recognition system made frequent errors. With the adaptive algorithm, the error rate decreased over time as more subject-specific data were incorporated. When compared to a non-adaptive system, the adaptive system reduced the number of errors by 32.9% [8.6%], mean [standard deviation]. This study demonstrates the potential improvements of an adaptive pattern recognition system over non-adaptive systems presented in prior research.

I. Introduction

Powered prosthetic knees and ankles provide joint power to lower limb amputees and assist them in completing a variety of ambulation tasks (e.g., walking on level ground, stairs, or inclines) [1,2]. These devices are typically programmed with different locomotion modes that change the behavior of the device (e.g., provide or dissipate power) as the user is completing the different tasks [3–6]. Finding a robust method for seamlessly and automatically transitioning the device between the different modes remains a challenge.

Pattern recognition algorithms have been proposed for automatically selecting the desired mode of the user. These algorithms can use kinetic and kinematic information from mechanical sensors embedded within the prosthesis to infer the user’s intent and transition the prosthesis into the desired mode [7–11]. One disadvantage to using pattern recognition is that a large amount of data must be collected to train the algorithm to learn user-specific patterns and recognize how a particular user completes the different mode transitions. To acquire these data, the user must complete a long and burdensome protocol where he or she uses the prosthesis to complete the mode transitions. Developing a pattern recognition algorithm that does not require the subject’s unique data to perform at a high level would eliminate the need for the demanding training protocol and improve the clinical viability of these devices.

A more robust pattern recognition algorithm could be developed by using training data collected from multiple users, obviating the need to collect training data from a novel user. However, previous studies have shown that these user-independent systems (i.e., those that are trained with data from multiple users but without the novel user’s unique data) have increased error rates, suggesting that subject-specific data is required for optimal performance [12]. One could address this limitation and improve user-independent systems by designing one that is adaptive. Specifically, an adaptive user-independent system could gather unique training data from the novel user as they are using the prosthesis to ambulate outside a clinical setting. This strategy would decrease error rates over time as the system automatically learns from unique subject patterns during daily use.

The objective of this study was to develop and evaluate an online adaptive pattern recognition system for a powered lower limb prosthesis. This analysis was completed with two able-bodied individuals using an adaptive system initially trained with data from a different user. The adaptive system was automatically updated with mechanical sensor data from the novel user as they were ambulating with the prosthesis in real-time. We hypothesized that users would experience fewer errors over time while using our adaptive system. Moreover, we highlight the benefits of using an adaptive system by comparing its performance to that of a system that was non-adaptive.

II. Methods

A. Adaptation Algorithm

Our proposed adaptive algorithm used supervised learning, meaning that all data used for training had to be paired with the correct class (in this case, the desired mode of the user). Thus, our adaptive algorithm had to meet the following requirements: 1) predict the desired mode of the user before critical transition points (e.g., heel contact, toe-off), 2) automatically label new patterns used to update the system with a class that matched the user’s intent, and 3) add the labeled pattern to the training dataset to update the system (i.e., updating parameters such as means, covariances, weights, etc.)

The first requirement of predicting the desired mode of the user is accomplished by classifying mechanical sensor information acquired before the user’s next step with the prosthesis, a process we term forward prediction. To accomplish the second requirement of automatic labeling of new prediction patterns, we use a separate pattern recognition system to classify mechanical sensor information acquired from the user’s entire completed stride. This process, which we term backwards estimation, classifies the executed gait pattern of the user. This has been shown to be an accurate strategy for labeling patterns that will be used to update the forward prediction system (Fig. 1) [13].

Figure 1.

Illustration of adaptive algorithm. A forward predictor (left) classifies mechanical sensor patterns from data before the stride and transitions the leg into the predicted mode. After mechanical sensor data from the entire stride is collected, a backwards estimator classifies this longer window of data and determines the executed gait pattern. The output of the backwards estimator then provides a mode label for the pattern of data acquired before the stride (right). The pattern of sensor data and its given label are used to update the parameters of the forward predictor.

Consider an example where the user’s desired mode is level walking. The forward prediction system classifies a pattern (P_FP) before the stride as level walking and the prosthesis is controlled in level walking mode. P_FP will also be used to update the forward prediction system, but requires a class label that should match the desired mode of the user. The backwards estimator classifies the gait pattern acquired from information from the entire completed stride (hopefully as level walking) and provides a class label for P_FP. The combination of P_FP and the class label can be used to update the appropriate parameters of the forward predictor.

Included in the design of the backwards estimator is the ability to provide a correct label for P_FP even when the forward prediction system misclassifies it (causing the user to take a stride in the incorrect mode). Consider again an example where the user’s desired mode is level walking but now the forward predictor incorrectly classifies P_FP as ramp descent. In this case, the user takes a step on level ground while in ramp descent mode. The backwards estimator can recognize this executed gait pattern and still provide a class label of level walking for P_FP. Thus, the label is correct because it matches the user’s desired mode.

B. Experimental Protocol

Two abled-bodied subjects completed the experiment, which was approved by the Northwestern University Institutional Review Board. Written and verbal consent was obtained from each subject involved.

Subjects wore a bypass socket (Fig. 2) that allowed them to ambulate with a prosthesis despite being able-bodied. The Center for Intelligent Mechatronics at Vanderbilt University designed the prosthesis used for this study [1]. A certified prosthetist attached the powered knee and ankle prosthesis to the subjects’ socket. Each subject had previous experience walking with the prosthesis and published strategies were used to tune the leg for each mode [4–6].

Figure 2.

Able-bodied subject with a bypass socket wearing the powered knee-ankle prosthesis.

Each subject participated in a session to collect training data for a forward predictor and a backwards estimator. This first session was designed to capture all relevant transitions between modes. Each subject completed tasks including standing, walking on level ground, stair ascent and descent on a 4-step and 3-step staircase, and walking on ramps. The experimenter used a key fob to trigger all mode transitions at specific points within the gait cycle (heel contact, toe-off, mid-swing, and mid-stance). The subjects were also asked to complete various activities while ambulating in the incorrect mode (e.g. walking down a ramp while in level walking mode). This was completed to provide data for the backward estimator so that it could recognize the true intent of the user if the prosthesis transitioned to the incorrect mode. Data from these experimental sessions were then used to train a forward predictor and a backwards estimator unique to each subject. We will refer to this session as the ‘offline session.’

In a second experimental session, the subjects completed a similar but shortened protocol to that of the offline session. In this session, an online forward prediction system trained with only the other subject’s data controlled the prosthesis. Therefore, each subject was a novel user because they were ambulating with a forward prediction system that was trained with data other than their own. It is important to note that the forward predictor sometimes made errors and transitioned the leg into the incorrect mode during this ‘online session’. As each subject was ambulating, the backwards estimator (also trained with the other subject’s data) labeled patterns used by forward predictor, and these patterns were then used to update the parameters of the forward predictor. In this case, class means and covariances were sequentially updated after every step with the prosthesis. The experimenter updated and saved the weights of the predictor periodically throughout the session. It should be noted that subject 2 completed five sets of a four-step staircase with a key fob instead of with an online predictor in the beginning of the online session. This was done to facilitate stair descent and to update the adaptive system with correctly-classified patterns. The backwards estimator still labeled the patterns from this offline set.

B. Signal Processing and Classifier Architecture

Kinetic and kinematic information from twenty-two embedded mechanical sensors were recorded at 500 Hz. These included joint angles, joint velocities, motor currents, and load applied through the prosthesis.

For forward prediction, data were segmented into windows of 300 ms before the aforementioned transition points. The mean, standard deviation, maximum, minimum, initial and final values of each mechanical sensor were calculated as features from each window [7]. The dimensionality of this feature set was reduced from 132 features down to 50 using principal component analysis [14]. The forward predictor was a dynamic Bayesian network, which incorporates the time history of the mechanical sensors into its predictions [10]. We used a mode specific classifier architecture for forward prediction. Thus, each mode had its own classifier that predicted transitions between modes. We used this architecture because it has been shown to improve performance for novel users [12]. The forward prediction system used in this study predicted transitions between the following modes: level ground walking, standing, stair ascent/descent, and ramp descent.

For backwards estimation, mechanical sensor data were segmented by strides (i.e., from one heel contact to the next heel contact). For each mechanical sensor, the same set of features as those used for forward prediction were also extracted from this stride window [13]. The dimensionality of this feature set was reduced from 132 features down to 13 using uncorrelated linear discriminant analysis (ULDA) [15]. The backward estimator used linear discriminant analysis (LDA) classifier to classify the executed locomotion mode at every heel contact. The classes of the backwards estimator included standing, level ground walking, ramp descent, stair ascent/descent, and various classes where the user was ambulating in the incorrect mode (e.g., walking down a ramp in level walking mode, completing stair descent one step at a time instead of step-over-step stair descent).

D. System Evaluation

To evaluate the performance of the adaptive system, we calculated the number of errors made by the forward predictor in the online session for each subject. We also calculated the number of mistakes the forward predictor would have made if the system were not adapted throughout the online session. Comparing the number of errors of both systems revealed the benefits of adaptation. It is worth noting that the non-adaptive system was not tested in real-time and that its response was determined offline.

We also determined the performance of the forward predictor before and after adaptation by testing the initial and final set of weights on the subject-specific data collected in the offline session. Error rates for this analysis are the pooled misclassification rates at the critical transition points of the prosthesis. Misclassifications were categorized as either steady-state or transitional misclassifications, where steady-state misclassifications occur when the prosthesis should not switch modes, and transitional errors occur when the prosthesis should switch modes.

III. Results

Both subjects initially had difficulty using the prosthesis to transition modes and completing the different ambulation tasks. For instance, subject 1 had difficulty initiating and completing step-over-step stair ascent/descent. The forward predictor of subject 2 would frequently miss transitions from level walking mode to ramp descent mode.

Fortunately, the backwards estimator correctly classified many of these steps and added these new patterns to the subjects’ training sets with the correct label. The result is that fewer mistakes were made over time (Fig. 3). If subject 1 had used a non-adaptive system, their predictor would have made 67 misclassifications. Instead, the adapted predictor made 49 misclassifications (a 25% reduction). The non-adaptive predictor of subject 2 would have made 41 misclassifications; their adapted predictor made 25 (a 39% reduction).

Figure 3.

Example of number of misclassifications made by one adaptive and non-adaptive classifier for subject 1 in the online session. The figure shows the number of misclassifications (marked with red circles) made by the heel contact classifier acting in level walking mode throughout the course of the online session. No red mark means that no misclassification occurred for that particular pattern. The top line of patterns shows the decisions that the non-adaptive classifier would have made, and the bottom line of patterns shows the decisions made in real-time by the online adaptive system.

The benefits of using the adaptive system were also observed when each subject’s final system (after all adaptation was finished) was tested on their unique dataset that was collected in the offline session. For transitional steps (when the leg should switch modes), the adapted predictor had a transitional error rate of 10.18% [3.80%], mean [standard deviation], whereas the initial predictor before any adaptation had a 22.92% [12.82%] transitional error rate. For steady-state steps, (when the leg should not switch modes) the adapted predictor had an error rate of 3.26% [2.39%], mean [standard deviation], whereas the initial predictor before any adaptation had an error rate of 3.19% [2.68%].

IV. Discussion

This preliminary study demonstrates how an adaptive pattern recognition system that learns subject-specific data can prevent errors that would have normally been made by a system that did not use adaptation. In this study, two able-bodied individuals walked on a powered knee-ankle prosthesis with an online adaptive pattern recognition system. The system was originally trained with data from the other subject, making each subject a novel user. Both subjects initially experienced frequent errors made by the forward predictor. Over time, the adaptive system learned to incorporate new subject-specific data from the user to retrain the algorithm during ambulation. As a result, the forward predictor made fewer errors over time, and made fewer errors than the non-adaptive system would have made.

The backwards estimator was able to correctly classify gait patterns when the prosthesis was in the correct mode and also in the incorrect mode. For instance, subject 2’s initial forward predictor would frequently misclassify the mode transition from level walking to ramp descent. This resulted in subject 2 taking their first step on the ramp in level walking mode. This step generated a unique gait pattern which the backwards estimator then correctly classified as a step where the prosthesis should have been in ramp descent mode. The adaptive system then took the pattern that was originally misclassified by the forward predictor, and applied a corrected label of ramp descent to update the predictor. Often, only one or two examples of a specific transition were required to teach the forward predictor how to correctly initiate the transition.

The performance of the adaptive system was also observed when the final predictor was tested on the datasets acquired in the offline session. The final adaptive system reduced error rates for transition steps by 12.74% [9.02%]. It is notable that this decrease in error rate was achieved even though both subjects took fewer steps in the online session than they did in the offline session. We would expect error rates to continue to decrease as more data from the current user are added to the training set.

This study has several limitations. First, the subjects in this study were able-bodied, Future experiments will investigate whether similar results can be found with lower limb amputees. Moreover, the forward predictors were particularly limited because they only had data from one individual. Clinical implementations of this system would likely start with a training set composed of data from many other individuals. Also, the current adaptive system did not ‘forget’ data from the other subject; instead, new data from the user was simply added to the training set, resulting in a dataset comprised of two individuals. Future implementations of the adaptive system will likely include a forgetting factor.

Lastly, although system parameters such as class means and covariances were sequentially estimated as the subjects were ambulating, our system did require that the experimenter manually and periodically update the weights of the forward predictor. This is because calculating the weights is computationally intensive and caused the leg to misbehave if done during ambulation. Future studies should investigate how often the system should be updated, and after which activities.