Body Machine Interfaces for Neuromotor Rehabilitation: a Case Study

Abstract

High-level spinal cord injury (SCI) survivors face every day two related problems: recovering motor skills and regaining functional independence. Body machine interfaces (BoMIs) empower people with sever motor disabilities with the ability to control an external device, but they also offer the opportunity to focus concurrently on achieving rehabilitative goals. In this study we developed a portable, and low-cost BoMI that addresses both problems. The BoMI remaps the user’s residual upper body mobility to the two coordinates of a cursor on a computer monitor. By controlling the cursor, the user can perform functional tasks, such as entering text and playing games. This framework also allows the mapping between the body and the cursor space to be modified, gradually challenging the user to exercise more impaired movements. With this approach, we were able to change the behavior of our SCI subject, who initially used almost exclusively his less impaired degrees of freedom - on the left side - for controlling the BoMI. At the end of the few practice sessions he had restored symmetry between left and right side of the body, with an increase of mobility and strength of all the degrees of freedom involved in the control of the interface. This is the first proof of concept that our BoMI can be used to control assistive devices and reach specific rehabilitative goals simultaneously.

I. Introduction

Spinal cord injury (SCI) causes a loss of motor and sensory functions below the level of lesion. Approximately 12,000 new cases occur each year in the United States [1]. For many of these individuals, especially those with a lesion at the cervical level, learning how to redirect their remaining motor functions is essential for controlling assistive devices in an optimal way. Computers and assistive devices, such as powered wheelchairs, are instrumental for them to interact with the environment and partially replace the lost functionalities. However, the use of these technologies is often challenging for the most impaired users, who need to reorganize their residual ability to efficiently generate commands and control signals. Difficulties are exacerbated by the fact that control strategies for these devices are often unintuitive, e.g. sip-and-puff control. A second challenge for people with SCI is the recovery of lost motor functions. A rich body of literature suggests that sustained sensory-motor practice promotes and facilitates plastic changes at the spinal cord following injury [2]–[4]. Most SCI survivors receive rehabilitation treatments from therapist in the form of intense physical exercise shortly after injury, when they are hospitalized. Despite these therapies promote motor recovery, they do not continue with the same frequency or intensity after release from the hospital. This is partially due to the fact that released patients do not have easy access to exercise equipment or to trained therapists despite widespread awareness of the importance of exercise for recovery [5]. Thus, it is clear that the current state of the art does not adequately address the issues of easy and intuitive use of external devices, and continued motor recovery.

The introduction of body-machine interfaces (BoMI) may provide a solution to these problems. BoMIs use the abundant number of degrees of freedom present in the human body to set control variables in low dimensional spaces [6]. Previous studies have demonstrated the ability of unimpaired and paralyzed participants (lesion between C3 and C6) to reorganize the coordination of high dimensional upper-body motions to control a cursor on a screen or a virtual keyboard [7]. The technology used to non-invasively record the upper-body movements was based on infrared video cameras (V100, NaturalPoint Inc., OR, USA) that tracked small active infrared markers [8], or inertial measurement units (IMUs) MTx (Xsens) placed on the shoulders [9]. In the present study, we have updated the IMU technology to use lower cost, wireless IMU sensors (YEI Technology, 3-Space Sensor™ Wireless 2.4 GHz DSSS).

All previous approaches with the BoMI focused on facilitating the use of an external device and not specifically on rehabilitation goals. Nevertheless, preliminary evidence [8] suggests that paralyzed SCI participants may obtain some physical benefits collateral to practicing with the BoMI, such as an increased range of motion and some strengthening of the shoulders. Thus, we hypothesize that the BoMI can be specifically programmed to engage the users in functional exercise aimed at movement recovery while simultaneously controlling the external device. In particular, it is possible to modify the parameters of the body-to-task mapping to either facilitate the performance of the task or, alternatively, to encourage exercising of degrees of freedom that are partially impaired, but not completely affected by paralysis. In this case study we addressed the specific issue of body-motion symmetry on a single SCI participant with an incomplete lesion at the C4 level. The participant demonstrated a marked reduction in the mobility of the right side of the upper body compared to the left side. Thus, we modified the parameters of the BoMI so as to increase the role of the right side and tested the effect of this alteration after an extended period of practice. The results of this preliminary study support the effectiveness of the approach in restoring a higher level of symmetric mobility.

While more extensive clinical validation is required to reach a solid conclusion, the present study is the first proof of concept that this type of BoMI may be effectively used not only for implementing efficient control in low dimensional spaces, but also exploiting the redundancy offered by the system to reach specific rehabilitative goals.

II. METHODS

A. The Body Machine Interface

To expand the portability of the system at a reduced cost without affecting the performance, we used 4 wireless and low cost IMUs manufactured by Yei Technology, 3-Space Sensor™ Wireless, see specifications in Table I. The interface is based on three main components: (i) 4 IMUs mounted on a customized upper-body vest and arm bands, in particular sensor 1 is placed on the left arm, sensor 2 on the left shoulder, sensor 3 on the right shoulder and sensor 4 on the right arm; (ii) transmission of body-motion signals via a wireless communication protocol to a dongle connected to a laptop that will process the transmitted data; (iii) mapping of these body motions into commands for moving a cursor on a screen by fitting the user’s residual mobility. Each IMU outputs 3 signals in real time: pitch, roll and yaw angles. We did not use the yaw angle because it is based on measurements from a tri-axis magnetometer that tends to drift or provide unreliable measurements in presence of strong magnetic fields. Accordingly, the IMU system generates an 8 dimensional signal vector, containing the output of all sensors. The BoMI operates by transforming this body-motion vector h̄ = [h₁, h₂, … , h₈]^T, into a lower dimensional control vector x̄ = [x₁, x₂]^T that is used to guide the movement of a computer cursor. This transformation is obtained via principal component analysis (PCA) [8]. In the initial session, the participant executed free-style upper-body motions for 1 minute. PCA was performed on these data to extract the 2 principal eigenvectors of the covariance matrix, a₁ = [a_1,1, a_1,2, … , a_1,8]^T and a₁ = [a_2,1, a_2,2, … , a_2,8]^T. These two eigenvectors are combined in a matrix A that generates the linear mapping from the body space to the cursor space (1):

$$\overline{x}=\left[\begin{array}{ccc}{a}_{1,1}& \cdots & a_{1,8}\\ a_{2,1}& \cdots & a_{2,8}\end{array}\right]·\overline{h}=A·\overline{h}$$ (1)

TABLE I.

IMU SPECIFICATIONS

	3Space Wireless (Yei Technology)
Orientation accuracy	±1 deg for dynamic conditions
Orientation resolution	<0.08 deg
Communication interface	USB, 2.4GHz DSSS Wireless

The two components of x̄ are the horizontal and vertical coordinates of the cursor, respectively. Moreover the BoMI features an intuitive user interface developed on a Simulink (MathWorks) platform, with a menu through which the user can easily navigate to access different tasks, and a “control menu” that the operator (experimenter or physical therapist) can use to properly modify the map in such a way to design exercises with specific rehabilitative goals.

B. Map Modification

The use of the PCA allows us to select not only the subspace that is most comfortable for the user to act upon, but also the degrees of freedom and coordination patterns that the user has more difficulty to operate. This gives us the opportunity to challenge the users in a rehabilitation exercise as they are carrying out a functional task. We can operate two different modifications.

The first is a modification of the input IMU signals. This is obtained by multiplying the input vector by an 8×8 selection matrix, S. S is a diagonal matrix where each diagonal value s_ii is set to 1 if we do not want to modify the weight of corresponding input signal h̄i, s_ii>1 if we want to increase it, 0<s_ii<1 if we want to decrease it. The second modification changes the contribution given by each IMU signal to each direction of the cursor space. This is obtained by pre-multiplying S by a 2×8 matrix, D. The final effect is a transformation of the mapping matrix:

$$A_{\mathit{new}}\leftarrow A\circ M$$

where ∘ indicates the Hadamard matrix product (each element of A is multiplied by the corresponding element of M) and M = D · S.

For example, if we want to give more importance to the two outputs of sensor 1 and 2 sensors we set:

$$S=\mathit{diag}([\begin{array}{cccccccc}{g}_1& g_2& g_3& g_4& 1& 1& 1& 1\end{array}])$$

with g₁, g₂, g₃, g₄ > 1.

If we want to give more control authority to sensor 2 on the vertical direction we will act on the 3^rd and 4^th elements, corresponding to the two channels of sensor 2, of the second row of D, corresponding to the Y cursor direction:

$$D=\left[\begin{array}{cccccccc}1& 1& 1& 1& 1& 1& 1& 1\\ 1& 1& d_{23}& d_{24}& 1& 1& 1& 1\end{array}\right]$$

with d₂₃, d₂₄ > 1.

Thus, we obtain the following matrix M:

$$M=\left[\begin{array}{cccccccc}{g}_1& g_2& 1& 1& 1& 1& 1& 1\\ g_1& g_2& g_3{d}_{23}& g_4{d}_{24}& 1& 1& 1& 1\end{array}\right]$$

C. Subject

One SCI survivor participated in the study after signing the informed consent approved by Northwestern University Institutional Review Board. He is a 35-year-old male, 90 days post injury, from the in-patient unit of the Rehabilitation Institute of Chicago. The level of lesion is C4 incomplete, resulting in a very poor control of the right part of his upper body, especially the arm, and a better control of the left side, in particular the left arm. No hand movements are present.

D. Experimental Setup and Protocol

The subject participated in one calibration session on day 1, where we created and customized the body-to-cursor map, A, followed by one reaching trial. The reaching trial consisted of 24 center-out reaching movements to 8 targets, equally distributed around a central target that appeared randomly on the screen. From day 2 to day 4, the user became acquainted with the interface. These familiarization sessions allowed us to analyze his upper body movements and to establish how to modify the map for achieving the rehabilitative goal. After day 4 we modified the map. This participant was using almost exclusively his left arm for controlling the interface. Therefore, we decided to act on the map so as to encourage him to engage also the other parts of the upper body. In day 5 and day 6 we changed the gains of the IMUs placed on the shoulders, sensors 2 and 3, by setting the corresponding coefficients in the matrix S. Then, from day 7 to day 11 we also acted on the coefficients of D. Combining the two transformations, we increased the authority of the right side of his body, the weak part, for the control of the cursor in the vertical direction, while reducing the authority of the left side. With these changes, the participant was induced to engage movements that were initially hard to execute. At the beginning and end of each session of the training program, the user performed a reaching trial. Between these trials he played virtual pong for 15 minutes. This is a game where the user controls the motion of a paddle that must hit a ball traveling across the screen and bouncing off the top, bottom and side walls of a court. Each time the ball bounces off the top wall the player is rewarded scoring one point.

At the beginning and at the end of the training, we executed a modified Manual Muscle Test [10] to characterize the strength and range of motion of the upper body of the subject. The test was performed while the participant was sitting in his wheelchair. The movements that we tested are reported in Table II. Each movement was evaluated with a number from 0 to 5, where 0=zero, 1=trace, 2=poor movement without gravity, 3=fair movement against gravity, 4=good, 5=normal. The maximum score for the scapula is 15, for the shoulder is 20, and for the arm is 10. In addition, we used a force transducer (Mark-10, force gauge MG series) to measure the isometric forces of the shoulder during movement in the upward, backward and forward directions.

TABLE II.

MOVEMENT EVALUATED WITH MMT

Body Part	Movement
Scapula	Elevation – Upper Trapezius
	Adduction – Rhomboids
	Abduction - Serratus Anterior
Shoulder	Flexion – Anterior Deltoid
	Abduction – Middle Deltoid
	Horizontal Adduction – Pectoralis Major-Clavicular
	Horizontal Abduction – Posterior Deltoid
Elbow	Flexion – Biceps Brachii
	Extension – Triceps Brachii

E. Data Analysis

Motor skill learning

To evaluate if the subject became more skilled at controlling the cursor through upper body movements we used two indicators: 1) Time to target, i.e. mean time elapsed before reaching the targets; 2) Normalized path length, i.e. mean length of the paths traveled by the mouse, divided by the length of the straight line from starting to end position.

Quality of movement and body use

To quantify the amount of motion performed by each instrumented part of the upper body we calculated mean value and standard deviation of the signals generated by the two channels of each sensor during two sessions: a) session four, the last session of the familiarization phase, and b) session eleven, the last session of the training phase.

Body contribution to mouse movement

We also wanted to isolate the contribution of the different body parts to the mouse movement. The first 4 elements of the body signal vector derived from IMUs on the left side of the body, while the last 4 from IMUs on the right side. Therefore we rewrite h̄ as the sum of left and right vectors:

$$\overline{h}={\overline{h}}_{\mathit{left}}+{\overline{h}}_{\mathit{right}}=\left[\begin{array}{c}{h}_1\\ \cdots \\ h_4\\ 0\\ \cdots \\ 0\end{array}\right]+\left[\begin{array}{c}0\\ \cdots \\ 0\\ h_5\\ \cdots \\ h_8\end{array}\right]$$ (2)

Substituting (2) in (1), one obtains

$$\overline{x}=A·{\overline{h}}_{\mathit{left}}+A·{\overline{h}}_{\mathit{right}}={\overline{x}}_{\mathit{left}}+{\overline{x}}_{\mathit{right}}$$

III. RESULTS

In the first four training sessions, the participant learned to proficiently control the cursor with his upper body using the map established during the calibration phase. During the reaching tasks of the familiarization phase, the cursor movement became faster and straighter, and the normalized path length (Fig.1a) and the time to reach the target (Fig.1b) decreased. During session one, the user performed only one reaching task, because in the first session we ran the assessment tests for evaluating his physical conditions and we customized the BoMI. This is the reason why in Fig.1 there is only one bar in correspondence to session one. In contrast, during the other sessions the participant was always performing two reaching tasks. When we looked at an example of trajectories in the 8 directions of the reaching, we noticed an improvement between session one and four (Fig. 2a,c) in the quality of the trajectories as they are straighter and it is easy to distinguish the 8 directions. On the contrary, if we look at the contribution of the two sides of the body to the cursor movements at the beginning and end of the familiarization phase (session four) (Fig. 2b,d), we observed that the participant - consistent with his impairment - almost exclusively consolidated the use of his left side of the body to control the cursor. In session five, we increased the gain of the IMUs placed on both shoulders, and in session seven we also increased the contribution given by the signals of the IMUs placed on the right shoulder and arm to the vertical direction of the cursor space, as described in the methods. The performance worsened in session five and it further decreased in session seven. However, in the last 5 sessions the subject learned how to control the cursor using the modified map, and both metrics decreased to a level comparable to the performance of session four (Fig. 1). When we looked at the contribution of each sensor to the cursor movements during the second reaching trial of the last session (Fig. 2f), we observed that the participant was using both sides of the body to control nearly orthogonal movement directions in the cursor space; the left and the right side controlled the horizontal and vertical direction, respectively. Also, the quality of the trajectories is comparable with that of session four.

Figure 1.

Reaching performance metrics. a) Mean normalized path length b) Time to reach the target. In dark blue is indicated the reaching 1 and in light blue the reaching 2 executed in every session. The dashed lines indicate when a change in the map occurred.

Figure 2.

Reaching trajectories and body parts’ contributions. The left panels show example reaching data (blue) in cursor space during the reaching trial of session 1 (a), the second reaching trial of session 4 (c) and the second reaching trial of session 11 (e). Only one trajectory for each direction of reaching is reported. The right panels of the figure (b), (d), and (f) show the body parts’ contribution to the cursor movements throughout the entire duration of the reaching task of the corresponding sessions. Left side in green and right side in red.

The data in the signals space confirmed the results highlighted in the cursor space. In Fig. 3 we report mean and standard deviation values of pitch and roll outputted by the IMUs placed on left and right arms and shoulders. A large standard deviation corresponds to a broad distribution of the data around the mean value, i.e. to a big movement. Zero degrees correspond to the initial position. At the end of the familiarization phase, session four, the left side of the body was moving the most. We observed high mean and standard deviation values for the pitch and roll angles in the left arm. On the contrary, the mean values of the sensors on the right body parts were closer to zero degrees, and their standard deviation was very small. During the last training session, all these values changed. In particular, the mean values of the right shoulder and arm were further from 0 compared to the end of the familiarization phase, and the corresponding standard deviations were substantially bigger.

Figure 3.

Reaching task in body space. Mean and standard deviation of roll and pitch angles measured by the IMUs placed on the upper body. The dashed line at 0 degree corresponds to the starting body posture. In the shades of blue are reported the values of the left side of the body. Dark blue corresponds to the body movement during the second reaching trial of session 4, and light blue during session 11. In the shades of red are reported the values for the right side of the body. Also in this case dark red corresponds to the second reaching trial of session 4, and light red to the second reaching trial of session 11.

Our preliminary data also show a positive effect of the use of the BoMI on the recovery of muscle strength and mobility of the upper body parts (Fig.4). The sum of the MMT values for each upper body district (scapula, shoulder and arm) after the training period increased, Fig. 4a. There was a general increase of this indicator for both shoulders and scapulae, there was a considerable increase in the MMT total score for the right arm from 0 to 3. We also decided to assess the force at the shoulders quantitatively. In Fig. 4b the total force score is reported. Consistent with the MMT scores, the force score increased in the two weeks period of training with our proposed protocol.

Figure 4.

a) MMT total score for the right and left scapula, shoulder and arm before (dark blue) and after the training (light blue). b) Total force score [N] at left and right shoulder measured with a force sensor (sum of the forces exerted by one shoulder in the three tested directions).

IV. DISCUSSION

Our proposed approach used our BoMI not only to facilitate the control of an external device, but also to test the possibility of achieving a rehabilitative goal. We had the participant practice and familiarize himself with the BoMI for the first four sessions, and he was able to learn to use it in a skillful way. Fig.1 shows that both metrics until session four decreased. Moreover, it is evident that there is a process of learning because the trajectories became more distinguishable and straighter. During these first four sessions, we gave the participant a map that is customized to his movement ability, making the use of the interface easy and intuitive. Thus, the subject moved his upper body in a way that was comfortable and simple. The left part of the body was the one that he was mostly engaging during the use of the interface, Fig. 3, and that contributed almost exclusively to the cursor movement, Fig. 2b, d. This is expected because of the impairment of the subject, incomplete lesion at C3 level, with a greater impairment on the right side than the left side. The rehabilitative goal in this case was to try to reestablish symmetry between the left and right upper body parts and encourage the subject to recruit also his right shoulder and arm while performing the tasks. We wanted to operate a gradual change of the interface to avoid subject’s disorientation. For this reason, we increased the gain of the IMUs placed on both shoulder during session five and six, and also modified the contribution of the different IMUs from session seven. We expected a decrease in the quality of the performance after the first change, and no improvement before the second change (session five and six), because the subject was practicing only for two days. Once the interface had been completely modified and was stable until the end of the practice (from session seven until session eleven), the subject showed again a learning process resulting in improvement of the performance. This is evidence of the fact that despite using a more challenging mapping, the subject could complete every session and improve his daily performance, Fig.1.

At the end of the training program we were able to see a great difference compared to the beginning of the practice in the subject behavior and in the contribution of his different body parts to the cursor movement. It is possible to notice how the participant modified his behavior over time (Fig. 3). He used the right part of his body that during the familiarization phase was poorly engaged, and reduced the use of the left side of his body. This resulted in a more balanced contribution to the cursor movement, Fig.2. After familiarization with the interface the left body part was almost exclusively contributing to the cursor movement. At the end of the training program both left and right sides gave a distinct contribution to the movement of the cursor.

Clinical test (MMT) and force measurements at the shoulders, gave use encouraging preliminary data. Both measurements showed a positive trend, resulting in an increasing of the upper body mobility. It is worth noting that these big changes, in particular in the MMT scores, could also be the result not only of the practicing of our BoMI, but also of the daily exercising sessions that the participant was attending with physical and occupational therapists at the Rehabilitation Institute of Chicago.

Overall, the results presented above provide a proof of concept of the use of the BoMI in the rehabilitation field. Engaging the user in functional and entertaining tasks while practicing the interface and changing the map in the proposed ways can be a novel approach to home-based rehabilitation treatments provided by portable and low-cost technologies.