Clinical detection and movement recognition of neuro signals

Abstract

Neuro signal has many more advantages than myoelectricity in providing information for prosthesis control, and can be an ideal source for developing new prosthesis. In this work, by implanting intrafascicular electrode clinically in the amputee’s upper extremity, collective signals from fascicules of three main nerves (radial nerve, ulnar nerve and medium nerve) were successfully detected with sufficient fidelity and without infection. Initial analysis of features under different actions was performed and movement recognition of detected samples was attempted. Singular value decomposition features (SVD) extracted from wavelet coefficients were used as inputs for neural network classifier to predict amputee’s movement intentions. The whole training rate was up to 80.94% and the test rate was 56.87% without over-training. This result gives inspiring prospect that collective signals from fascicules of the three main nerves are feasible sources for controlling prosthesis. Ways for improving accuracy in developing prosthesis controlled by neuro signals are discussed in the end.

INTRODUCTION

Human limbs are complex, delicate systems. Their reconstruction, both for appearance and for function compensations, proves to be complicated projects. With the development of science and technology, both researchers and disabilities hope to have artificial limb much more perky, comfortable, dexterous and effective. Although thousands of prosthetic limbs are commercially available worldwide, most of them, from mechanical styles to electric styles, from motion controlled to EMG, or voice controlled, can’t meet the needs of users because of their poor performance properties. Degrees of freedom (DOF) that can be achieved are normally less than five (Lin et al., 1998), far from practical needs because of the insufficiency of independent information channels. Control accuracy is not high enough because of signal sources’ poor duplication and their sensitivity to outside disturbance. Usually, instead of using prosthetic devices as parts of their body, the disabled seem to be operating machine systems.

The above deficiencies pose a challenge to improve the properties in prosthesis design. More effective interface devices, new signal processing technologies, pattern recognition methods and intelligent control strategies have been explored and achieved much. However, attention is not being paid enough to possible signal sources for limb control.

Researches in neurophysiology show that neural systems are reparable, not only adaptable and compensatory to outside stimulations, but also have the ability of recovery and reconstruction after impairment, both in structures and in functions. Moreover, the patterns of neural activity repeat very well because they are indifferent to tiredness and interruption. Researches also reveal that electrode chips with holes made from silicon may establish reliable connections with neural cells if they are tied with amputated neural fibers (Kovacs et al., 1987), and can get stable and clear electric recordings on continuous discharges from neural fibers (Weber et al., 1999). Such researches make it possible to use neural activity as the signal source to control prosthesis.

Now, many studies have identified the relationships/associations between the discharge patterns of neural activity and arm movement parameters (e.g. velocity, direction and force). Humphrey (1972) found that exact forearm movements can be predicted by weighted sum impulse vectors from a group of cortical motoneuron recordings. Schwartz et al.(1988) discovered that the rate at which a neuron fires in the motor cortex determines the direction the associated muscle will tend to move, and that hand velocity can be predicted online by impulses from cortical neural cells (Schwartz, 1994). Burrow et al.(1997) successfully established the maps between cortical signals and forearm movements.

These achievements make it possible to develop prosthesis controlled by neuro signal. Chapin et al.(1999) successfully trained rats to position a robot arm to obtain water by pressing a lever with simultaneously recorded motor cortex neurons. Wessberg et al.(2000) achieved long-term control of complex prosthetic robot arm, both locally and through the Internet, by simple real-time transformations of neuronal population signals derived from premotor, primary motor and posterior parietal cortical areas in primates.

All the above works derived neuro signals with cuff electrodes, with the subjects being mainly animals instead of humans. Professor Chen Zhong-wei, academician of the Chinese Academy of Science and Engineering, proposed detection of the nerve signals by clinically implanting microelectrodes into the fascicule of three main nerves (median nerve, radial nerve and ulnar nerve) in the human being’s upper extremity. The development of microsurgery in China (Chen, 2002) and successful experiments (Zheng et al., 2002) on neuro signal detection with self-made intrafasicular electrodes on rabbits ensured that such idea is feasible, has sufficient fidelity, and is without any infection.

This paper presents a clinical experiment on neuro signal detection achieve with microsurgery. Initial analysis on features under different actions was conducted and movement recognition on samples of neuro signals was attempted. SVD features extracted from wavelet coefficients were used to predict an amputee’s movement intentions with neural network classifier. The whole training rate was up to 80.94% and the test rate was 56.87% without over-training. Ways for improving accuracy in developing prosthesis controlled by neuro signals are discussed in the end.

NEURO SIGNAL DETECTION

The self-made intrafascicular electrode was made from 95% platinum and 5% iridium with diameter of 60 μm, and insulated with 5 μm Teflon. Before the experiment, approximately 10 mm of insulation at the distal end of the electrode were removed by heating to prepare the electrode for implanting in the fascicle, and 1 mm of insulation at the proximal end was removed to allow for attachment to recording equipment. Then the electrode was penetrated through a silastic column with diameter of 2 mm and length of 10 mm and bounded together with cyahoacrylate. Here the electrode is not attached with a hard tungsten needle in order to lessen the damage to the tissue.

The subject was male, 31 years old, with left upper limb amputated two years ago. Two weeks neuro rehabilitation was implemented before the experiment to enhance the stability and reliability of neuro signals.

After full anaesthesia, the upper limb’s median nerve, radial nerve and ulnar nerve were extricated for about 5 cm length; then two motor fascicules of each nerve were selected to implant the intrafascicular electrode directly by microsurgery into the fascicle of the nerves, as shown in Fig.1. Another electrode was placed outside the fascicle and parallel to the intrafasicule electrode and used as reference electrode. The signal was recorded with an EMG Instrument “Paseidon NDI-500” with sampling frequency of 5 kHz.

Fig. 1.

Experiment on detection and recording of neuro-information (a) Electrodes were implanted into fascicule; (b) Sectional view of radial fascicule detection; (c) Sectional view of ulnar fascicule detection; (d) Sectional view of median fascicule detection; (e) Neuro signal detection; (f) Neuro signal recording

After reliable connection had been established between the electrodes and the fascicule, the subject was waken up and did actions such as hand open/close (HO, HC), wrist’s pronation/supination (WP, WS) and flexion/extension (WF, WE), altogether 6 movements under control of consciousness. Then the neuro signals were recorded real-time for further processing.

INITIAL FEATURE ANALYSIS

Each sampled signal had the length of 100 ms (500 points), altogether 32 groups of data were recorded, in which 26 groups were measured under action condition, and the other 6 groups were under relaxation condition. Each group had 4 channels, a total of 104 nerve signals were recorded. Fig.2 gives a typical example of neuro signals recorded from the fascicule of radial 2 under hand open action (to avoid losing information, the signals were not filtered further).

Fig. 2.

A sample of neuro signal recorded from fascicule of radial 2 under hand open action (a) Time domain; (b) Power spectral density (PSD); (c) Wavelet analysis; (d) Pseudocolor of coefficients

Initial analysis of the features of neuro signals showed that (Zhang et al., 2004):

1. The average absolute value of signals from the radial fascicule was 5.5 μV, with standard deviation of 0.8 μV, while values of signals from the ulnar fascicule and medium fascicule was 2.5±0.4 μV and 2.2±0.3 μV respectively. Further neurophysiology and physiologic anatomy research showed that the radial signals detected from the implanted position were almost pure motor signals, while ulnar fascicular signals and medium fascicular signals were mixed with a lot of sensational signals. This showed that motor signals are more intensive than those of sensational signals, at least in the detected position.

2. The power spectrum of the fascicular signals was distributed almost throughout the range of [0, 2.5] kHz, wider than that of myoelectricity. Perhaps the sampling frequency should be increased in the next experiment.

3. When an action occurs, the radial signals from electrode 1 and 2 differ little, but signals from the ulnar fascicule and medium fascicule differed obviously; the difference was probably caused by the sensational information from receptors in the ulnar nerve and medium nerve, not by the motor information coming from the brain.

4. The high-frequency component increases as the applied force increases, but the signal intensity differs little. This showed that the discharge rate of the neuro impulse had close relation with the applied force.

5. Antagonism, the typical feature of myoelectricity, was not found in neuro signals, either in different nerves or in different fascicules of the same nerve.

MOVEMENT RECOGNITION

In the following part, movement recognition was tried to test whether the detected fascicular signals (samples) are suitable for prosthetic control. Since they are collective and continuous, the same classification method as that for myoelectricity was adopted.

Signal processing

Considering that the samples were not enough, segmentation was used first. Each signal was divided into 5 segments with 128 points, where 35 points overlapped each other, as shown as Fig.3.

Fig. 3.

Illustration on segment sampled data

Thus, 25 samples were obtained for every HO, WE and WF; 10 samples for each HC and WP, and 20 samples for WS. Then, white noises with 5% signal noise to ratio (SNR) were added to increase the samples 10 times for each action, so that 250 samples were available for HO, WE and WF, 100 samples were available for HC and WP, and 200 samples were available for WS. From them, 3 to 5 were randomly selected as training samples, and the other were used as test samples.

Signal analysis

Signal analysis aims at separating information relevant for the recognition task from irrelevant information (e.g. sensation or noises) and at reducing the amount of data presented to the neural network classifier. Considering the on-time requirement and flexibility, wavelet transform (WT) was selected instead of Wigner distribution or short time Fourier transform because of its local and multffi-scaled outlook as well as its sensitivity to breaks.

Wavelet decomposition can be realized by the following Mallat Algorithm (Mallat, 1989):

(1)

where f is the signal to be analyzed, with length N, c _k ^j and d _k ^j being scaling coefficients and wavelet coefficients under scale j; h¯_n and g¯_n being the pulse response of conjugate mirror filters.

In wavelet analysis, different basis functions may be suitable for different signals, and appropriate selection of the wavelet basis for signal representation can result in maximal benefits. It is reasonable to think that if a wavelet contains enough information about a signal to be represented, the wavelet system can be simplified in terms of the level of required resolution, which reduces the computational complexity of the problem to be implemented.

In general, wavelet basis selection is related both with the signal to be analyzed and the implementation. As for neuro signal classification, the index for basis selection should be the representation effectiveness of selected feature vectors after wavelet transformation. However, this method is not suitable here since the original samples were not enough to provide convincing results. One simple way is to choose a basis available after some comparison, although such a result is not optimal. In this experiment, Daubechies 4 wavelet was selected by comparing the decomposition level required while keeping the energy as much as possible.

Recursive applications of the above Mallat algorithm led to the decomposition of the neuro signal into a matrix of sequences (Pittner and Kamarthi, 1999), as shown in Fig.4 (here three scale decomposition is adopted). The shadowed part is filled with zeros.

Fig. 4.

Wavelet analysis on neuro signal and its coefficients matrix

Feature extraction

The objective of feature extraction is to produce a suitable representation of neuro signals for movement recognition. Many methods can be used to form a feature vector according to the above matrix results (Zhang et al., 2003). Here average absolute coefficients, scaling energy, maximum coefficients, singular value decomposition (Zhang et al., 2002) and 4-order autoregressive (AR) coefficients are used for comparison.

Average absolute coef.:

Scaling energy:

Scaling maximum:

SVD:

AR coefficients:

where c_jk is the kth wavelet coefficient in scale j, λ(C ^T C) means to calculate the characteristic roots of the matrix C ^T C (C is the coefficient matrix formed in the above wavelet decomposition). x_n is the signal to be analyzed, a_k is the kth coefficients of AR model, e_n is white noise, and p is the total orders.

Thus, after feature extraction from signals of radial 2, ulnar 1 and medium 1, five groups of vectors with length of 4×3=12 are available. To compare their ability in representing neuro signals, the cluster-separation index (CSI), or Davies-Bouldin index (Davies and Bouldin, 1979), a measurement that is related to the performance of the linear Fisher Discriminant classifier of pairwise clusters, is introduced here:

(2)

Where K is the total classes, R_ij is an index that measure the similarity between class i and j, S_i and S_j describes the scatter degree of class i and j, D_ij is the average distance between i and j. They can be calculated as follows:

(3)

(4)

Here N_i represents samples of class i, x_j is input sample, m_j is the average of input of class i.

The CSI gives the overlap degree between different classes, and has been used widely in classification problem (Kermani and Wheeler, 1992). The smaller the CSI is, the more the classes separate easily.

The CSI value of each feature vectors is shown in Table 1.

Table 1.

Cluster separation index for features

Features	AVG	ENER	MAX	SVD	AR
CSI	26.1028	39.3547	15.2584	8.8363	11.2392

Clearly, as samples of this experiment are concerned, SVD features are better for neuro signals representation than other features, so they were selected as the inputs for further neural network classification. Fig.5 is a scatter plot of two SVD features from radial 2 under six different movements (white noises are not added here).

Fig. 5.

Scatter plot of SVD feature 1 and 2

It can be seen that the separation ability between these two features are not good enough, overlap exists seriously, especially among WF, HO, WE and HC actions. This indicates that the subject duplicates is unsatisfactory in acting the same movement, and should be trained for more time to keep well action duplication before experiments.

Classification analysis

Considering the features cluster not well enough, neural network with strong nonlinear mapping ability is used here as classifier. Multiple layer perception (MLP) with structure of 12-12-6 is used and genetic algorithm is adopted instead of BP algorithm to avoid local minimum. The feature values are normalized to [0, 1] before they are used as inputs. 0.9 and 0.1 are used to represent two different situations that belong to the class or not belong to the class. The population is 50, cross rate 0.6, mutation rate 0.4. Evolving strategy is selecting 20 chromosomes with best fitness from children to replace parents that have worse fitness. After 200 generations, the final training error of the best gene is up to 0.0418, the training curve can be seen in Fig.6. Table 2 gives the training error and testing error.

Fig. 6.

Train curve for neuro-signal classification

Table 2.

Accuracy of neuro signal recognition

	HO	HC	WE	WF	WP	WS	Average
Training rate	86%	87.33%	86.67%	72.50%	75%	76.33%	80.94%
Test rate	40%	43%	65%	62.50%	75%	72%	56.74%

It can be seen from Table 2 that the average recognition rate is encouraging, though they are not satisfactory for prosthetic control. Reasons that cause the low recognition rate may be as follows:

1. This experiment involves the human body, and the risk is very high because of the well-known reasons. For the sake of safety, the experiment does not last long, so the sampled data were somewhat insufficient, and possible abnormal samples are also used as classification without excluding them first.

2. Two years had passed after the subject’s amputation. Although half a month’s rehabilitation training was implemented before the experiment, the duplication of movements were still not satisfactory. This made the features scatters not well for high recognition rate. More rehabilitation time is needed before the next experiments.

3. As pointed out before, sensational signals are mixed in ulnar and medium fascicule. Obviously, they cannot be neglected and can greatly reduce the recognition accuracy since when the same action occurs, the sensational information may diverse greatly.

Clearly, these three disadvantages can be fairly improved, though they are inevitable to some extent. So, it can be said that developing flexible prosthesis with multi DOF controlled by neuro signal is quite feasible. Of course, in future, the through-the-skin wire should be replaced by a radio link connecting the fully implanted electrodes with the external decoding device.

DISCUSSION

This experiment on human body gives confidence that collective signals from the fascicule of the three main nerves are useful sources providing enough information for prosthetic control. Since neuro signals have so many advantages, as pointed out in the above discussion, it can be concluded that an ideal way for developing prosthesis is to obtain control commands by establishing mapping between the movement intentions and the patterns of neuro signals from intrafascicular electrodes clinically implanted into motor nerves by microsurgery technique. However, in developing commercially and flexible neuro prosthesis, besides more experiments on human body being needed, further research on the relations between neuro activity and movement intentions, ideal human-machine interface and controlling strategy are all indispensable.