Upper Body-Based Power Wheelchair Control Interface for Individuals with Tetraplegia

Abstract

Many power wheelchair control interfaces are not sufficient for individuals with severely limited upper limb mobility. The majority of controllers that do not rely on coordinated arm and hand movements provide users a limited vocabulary of commands and often do not take advantage of the user’s residual motion. We developed a body-machine interface (BMI) that leverages the flexibility and customizability of redundant control by using high dimensional changes in shoulder kinematics to generate proportional controls commands for a power wheelchair. In this study, three individuals with cervical spinal cord injuries were able to control the power wheelchair safely and accurately using only small shoulder movements. With the BMI, participants were able to achieve their desired trajectories and, after five sessions driving, were able to achieve smoothness that was similar to the smoothness with their current joystick. All participants were twice as slow using the BMI however improved with practice. Importantly, users were able to generalize training controlling a computer to driving a power wheelchair, and employed similar strategies when controlling both devices. Overall, this work suggests that the BMI can be an effective wheelchair control interface for individuals with high-level spinal cord injuries who have limited arm and hand control.

Introduction

High-level spinal cord injuries (SCIs) can result in severe motor deficits including weakness and uncoordinated movements. Specifically, injuries to the spinal cord at the cervical level often result in tetraplegia, or a loss of motor function that affects all limbs [1]. Many individuals, however, retain some movement, which can be used to control assistive devices such as power wheelchairs [2]. The ability to safely self-operate a power wheelchair dramatically increases quality of life and promotes independence for individuals with limited mobility [3 – 5]. The majority of power wheelchair users rely on a hand-controlled joystick as the method by which they control the movement of the power wheelchair [6], however many individuals with cervical spinal cord injuries may have limited arm and hand control or coordination. This can result in either a decreased ability to control the power wheelchair or the need to use alternative controllers. The organization of the spinal cord is such that the level at which motor neurons leave the spinal cord mirrors the location of the muscles they innervate. Motor neurons that innervate the distal muscles of the upper body (arms and hands) project out of the spinal cord inferior to the nerves that innervate the proximal muscles of the upper body (neck and shoulders). Individuals who do not have sufficient arm and hand control to use a joystick may have substantial shoulder and neck movement [2]. While there is not a direct link between neurological injury level and precise functional ability [7, 8], generally, individuals with spinal cord injuries between the C2 and C5 levels could generate coordinated shoulder movement but would likely have limited hand control.

For individuals whom do not have sufficient arm and hand control to use a joystick, there are a number of commercially available alternative power wheelchair controllers, the most prevalent being the sip-and-puff and head array [6]. While these controllers provide a means of transportation for individuals who cannot use a standard joystick, they have several limitations. First, the majority of these controllers provide the user with only a limited set of discrete commands. Users select from a set of four commands: drive forward, drive backward, turn left, and turn right. Users can only control the speed in incremental steps, and they cannot smoothly combine commands to execute more complex maneuvers. There are current commercial devices that provide users with a full set of proportional commands [9, 10], however, these controllers often have a predefined method of operation. Users must learn to conform to the controller. After it is set, the controller is not flexible to the user’s residual motor function. Similarly, users do not have the opportunity to utilize much of their remaining mobility, and instead rely solely on their head, neck, or tongue to control a power wheelchair. These shortcomings limit the usability of power wheelchairs and are some of the reasons that power wheelchair users report difficulty or inability to perform daily maneuvers [6].

There has been recent progress in the field of non-invasive human machine interfaces as a means for power wheelchair control [11 – 16]. Specifically, there are numerous electroencephalography- (EEG) and electromyography- (EMG) based wheelchair controllers. The major advantage of these systems is that they do not require any residual movement, as they rely solely on neural activity to generate wheelchair control commands. This type of controller would be ideal for extreme cases of paralysis such as locked-in syndrome or the most advanced stages of progressive conditions such as amyotrophic lateral sclerosis. Many of these interfaces, however, similar to the commercially available alternative wheelchair controllers, do not provide the user with proportional control. The user simply specifies the direction of motion, typically in one of the four cardinal directions, but does not have proportional control of the speed of the wheelchair [12]. This results in a more limited set of possible maneuvers. Additionally, while brain activity can be detected in slight anticipation of overt body motions, the rate of information transmission of non-invasive brain computer interfaces is limited by the time needed to process signals and classify brain activities and ranges currently from 0.05 to 0.5 bits/second [17, 18]. In contrast, recent studies have shown that classification of body movements can be achieved on a much faster time scale of about 5 bits/sec [19]. This inhibits the user’s ability to make fast changes, limiting practical use in crowded and rapidly changing environments. While there has been progress in shared control algorithms that allow for a more continuous and complete vocabulary of commands [13] as well as increased safety measures [20] users may desire to have complete control of the wheelchair and not rely on the consistency of noisy external sensors. Tongue based wheelchair controllers would also be ideal for individuals with very little residual motion. However they may hinder communication and present aesthetic and practical issues that can be avoided if the user regains some peripheral functional ability [9, 10]. Perhaps most importantly, tongue- and head-based wheelchair controllers, do not provide a way to significantly engage the residual upper-body mobility of their users.

To overcome some of the limitations of alternative power wheelchair controllers for individuals with high-level spinal cord injuries, it may be advantageous to develop a power wheelchair control interface that maximally leverages the remaining residual body motions [6, 21 – 24]. In fact, previous research has shown that individuals with cervical spinal cord injuries are able to continuously control the location of a two dimensional computer cursor to complete several tasks using only small shoulder movements [25], [26]. The underlying principle behind this body-machine interface (BMI) is to use dimensionality reduction techniques to map high dimensional measurements of shoulder movement to the state of a low dimensional controller. Principal component analysis (PCA) on random shoulder movements reveals movements that account for the most variability while remaining orthogonal [27, 28]. In other words, PCA allows us to identify two independent movements that are commonly executed by the user and can be continuously mapped to any two dimensional control commands [29, 30]. This work has formed the basis for a control interface that is fully customizable to the remaining upper body motion for individuals with high-level SCIs, which is highly dependent on the nature and location of the injury.

This type of interface has been shown to be effective for controlling a computer cursor to complete a variety of tasks including center out reaching, playing virtual ping-pong, or playing cards [31]. However, little work has been done to investigate whether users can learn to use this type of interface to control a power wheelchair safely and effectively. There exist many nontrivial differences between driving a power wheelchair and controlling a computer cursor, such as the need to stabilize and to compensate for external motion as well as the nature of the feedback (increased visual load and vestibular feedback). Additionally, little systematic work has yet been done to determine if users would employ similar control strategies to complete such drastically different tasks. Here, we investigated not only whether users can learn to use small shoulder movements to accurately control a power wheelchair but also how similar their control strategy for driving is to the strategy for controlling a computer cursor.

We compared power wheelchair driving performance using the BMI to performance using a traditional joystick, which was taken to be the gold standard for manual control. We expected that participants would be able to successfully control the power wheelchair using the BMI. However, since the participants with spinal cord injuries enrolled in this study had significant arm function and used a joystick to control their wheelchairs daily, we also expected that their performance using the BMI might be slightly worse than the performance using the joystick. We estimated based on regression analysis what control strategy users employed when driving the wheelchair and when controlling a computer cursor. We used this approach to test whether the subjects formed a single representation or two distinct representations to perform two functionally different tasks.

Methods

The Body-Machine Interface (BMI) described here uses high dimensional upper body movements to control the speed and direction of a power wheelchair. Specifically, inertial measurement units (IMUs) were used to detect and measure small shoulder movements which were mapped to power wheelchair control commands for individuals with spinal cord injuries. Two IMUs were placed on each shoulder of the participant to maximally capture shoulder movement. The sensors were attached to a vest that was worn above the participants clothing. The IMUs were placed on the same location on the vest for each training session. An additional sensor was placed directly on the wheelchair to cancel out IMU measurements that could be attributed to the motion of the wheelchair. This allowed us to separate intended shoulder movements from unintended movements resulting from bumpy or uneven surfaces. The IMUs used in this study were Xsens MTx sensors (Xsens Technologies B.V., Netherlands), which use tri-axis accelerometers, tri-axis gyroscopes, and tri-axis magnetometers to estimate the pose of each IMU. From each IMU, we continuously recorded changes in the roll and pitch. We did not record changes in yaw because the yaw measurement exhibited substantial drift, and was heavily influenced by changes in the heading of the wheelchair. Also, the yaw measurement is the only angle that relies on accurate magnetometer readings, which are influenced by changes in the local magnetic field from the wheelchair’s motors. From the roll and pitch of the four IMUs, at each sample time, we obtained an 8-dimensional vector (the “body vector”). IMU samples were taken at 50 Hz.

Power wheelchair operation by joystick typically relies on two continuous control signals [32]. Specifically, the control commands consist of a translational (C_T) and rotational (C_R) component. The translational component controls the forward speed of the power wheelchair, driving both wheels at the same angular velocity. The rotational command controls the turning speed of the wheelchair, driving the wheels at equal angular velocities but in opposite directions. The two control commands are sent to the power wheelchair as independent analog voltages, where the level of the voltage signal is directly related to either the forward speed or turning speed of the power wheelchair. By having the ability to independently select from a continuum of control commands, a power wheelchair user can execute a broad vocabulary of maneuvers.

For the BMI described here, the movement of the upper body was used to define the wheelchair command as a continuous two dimensional vector containing the translational and rotational command voltages. To achieve this, we used dimensionality reduction techniques, similar to those described in [25, 29, 33]. Specifically, we used PCA to decompose the high (8) dimensional shoulder movements into orthogonal components that best describe the structure of the variability of random shoulder movements [27, 28]. During initial calibration, users performed a 1 minute “dance” where they were instructed to perform random shoulder movements within a range of motion that avoided extremes and uncomfortable movements. PCA was performed on the IMU measurements during this “dance” to rearrange the signals into eight principal components ordered by decreasing variance. We then used the two principal components that accounted for the highest percentage of variance in the “dance” to construct a forward map (A; 8×2 matrix). The body vector (h) was then linearly mapped to the control vector (p) using A, according to equation 1.

$$p=\left[\begin{array}{c}\hfill p_1\hfill \\ \hfill p_2\hfill \end{array}\right]=\left[\begin{array}{cccc}\hfill a_{11}\hfill & \hfill a_{12}\hfill & \hfill \dots \hfill & \hfill a_{18}\hfill \\ \hfill a_{21}\hfill & \hfill a_{22}\hfill & \hfill \dots \hfill & \hfill a_{28}\hfill \end{array}\right]\left[\begin{array}{c}\hfill h_1\hfill \\ \hfill h_2\hfill \\ \hfill ⋮\hfill \\ \hfill h_8\hfill \end{array}\right]=Ah$$ (1)

The individual components of A (a_ij) represent the weighting of the ith IMU measurement on the jth degree of freedom for the controller. This method maps high dimensional shoulder movements to low dimensional control commands. A single calibrated map A was used for all sessions as learning was robust to small changes in the locations of IMUs. The components of p, were used to set the speed (v) and the angular velocity (ω) for the wheelchair.

$$\nu =K_{\nu }{p}_1$$ (2)

$$\omega =K_{\omega }{p}_2$$ (3)

K_v and K_ω are constant conversion factors between the components of p to speed and angular velocity and have the units 1/s and deg/(m*s) respectively Wheelchair motion can then be described by

$${\left[\begin{array}{c}\hfill x\hfill \\ \hfill y\hfill \\ \hfill \theta \hfill \end{array}\right]}_t={\left[\begin{array}{c}\hfill x\hfill \\ \hfill y\hfill \\ \hfill \theta \hfill \end{array}\right]}_{t-1}+{\left[\begin{array}{cc}\hfill \text{cos}(\theta )\hfill & \hfill 0\hfill \\ \hfill \text{sin}(\theta )\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right]}_{t-1}*{\left[\begin{array}{c}\hfill \nu \hfill \\ \hfill \omega \hfill \end{array}\right]}_{t-1},$$ (4)

where [x y θ]_t^T is the state of the wheelchair (2D location in space and heading) at time t. The components of p were continuous variables that could take on any positive (drive forward or turn left) or negative (drive backward or turn right) value depending on the configuration of the shoulders. This allowed the wheelchair to execute numerous maneuvers at different speeds. In addition to power wheelchair control, the principle described here can be used to control any system with two degrees of freedom such as a computer cursor [31] or the joint angles of a two-link planar robot [34].

In order to effectively control the interface, users must learn to solve the inverse problem Bp = h, where B is a right inverse of A (AB = I₂), p is the desired control vector and h is a (8-dimensional) body configuration that achieves this desired (2-dimensional) control vector. Because there are more degrees of freedom in the body space (the space spanned by h) than there are in the control space (the space spanned by p), the interface is over actuated. Thus there are infinite right inverses of A that can be used to solve this problem or infinite body configurations that map to the same control commands. Prior research suggests that users learn one of these right inverses when operating the interface [29]. To estimate what inverse users learn, we used least squares regression to obtain B̂ from a collection of N body vectors, H = [h⁽¹⁾, h⁽²⁾,…, h^(N)] and N control vectors, P = [p⁽¹⁾, p⁽²⁾,…, p^(N)].

$$\hat{B}=H{P}^T{(P{P}^T)}^{-1}$$ (5)

We considered B̂ to be the maximum likelihood estimated of the control strategy employed by the users.

To facilitate power wheelchair driving, real time feedback was displayed to users. The visual feedback was delivered via a small screen that was mounted to one of the armrests of the wheelchair (Figure 1). On the screen, the position of a cursor reflected the current state of the control command. The cursor moved the same as the tip of a joystick would. When the cursor was in the center of the screen, the power wheelchair was stationary. As the translational speed increased (forward) or decreased (backward), the cursor would move up or down respectively. Similarly, as the rotational command changed to cause the power wheelchair to turn left or right, the cursor would move left or right respectively.

Figure 1.

Diagram of the Body Machine Interface (BMI). Shoulder kinematics are recorded by inertial sensors, processed by an on board computer and commands are sent to the power wheelchair

Several additional considerations were taken into account to ensure that the interface provided a safe and efficient means of transportation. First, the average body position during the “dance” was set to correspond to the resting state of the wheelchair (both control commands were equal to zero). This guaranteed that it would be easy to stop the wheelchair by moving to a comfortable location. Also, this ensured that maneuvers in all directions would be equally easy for users. To ensure that users could not make unsafe maneuvers that might result it crashes or tipping, the wheelchair control commands were confined such that maximal rotational commands could not be achieved while there was a maximal translational command and vice versa. Specifically, the control commands were constrained such that the magnitude of the control vector did not exceed a certain value. If the body configuration resulted in a control vector outside this constraint, the individual control commands were lowered until the magnitude of the control vector fell within the appropriate range, while maintaining the same ratio between the individual commands. This is analogous to how most joysticks work. Lastly, a dead zone was enforced so that if the body deviated only slightly (15% of the maximal movement) from the mean posture along each principal component independently, the component of the control command mapped by that principal component would remain neutral. This ensured that it was easier for users to maintain each control command at zero independently and thus allowed users to stop with ease, as well as to easily execute singular maneuvers such as driving straight or turning in place.

Participants

Three individuals with spinal cord injuries (S1, S2, and S3) were enrolled in the current study. Specific injury levels, age, and the time since injury can be seen in Table 1. Generally, all participants had suffered injuries at the cervical level of the spinal cord that resulted in limited upper limb mobility and active range of motion as well as complete paralysis of the lower body and trunk. Despite this, all participants had sufficient shoulder movement, in that they could generate at least two independent movements that could be reliably recorded by the IMUs. All participants were experienced power wheelchair users and used hand controlled joysticks as their control method. No subject had any prior experience using the BMI. Three additional unimpaired control participants (age-, and gender-matched) that had no previous experience with the BMI or power wheelchairs participated in a second experiment. Participants with spinal cord injuries and unimpaired participants provided informed consent approved by the Northwestern University Institutional Review Board.

Table 1.

Subject Injury Details

	Injury Level	Age	Time Since Injury	Current Wheelchair Control Method
S1	C6	59	2 years	Joystick
S2	C6	41	8 years	Joystick
S3	C5	30	10 years	Joystick

Experimental Protocol

A schematic of the experimental design can be seen in Figure 2. The three participants S1, S2, and S3 partook in 24 training sessions that spanned four months. Sessions were biweekly and lasted roughly 1.5 hours. During each training session, participants used the BMI to control a computer interface. For this, the control vector was mapped to the two dimensional coordinates of a cursor on a screen [31]. Moving the cursor up and down corresponded to the same movements that would later cause the wheelchair to go forward and backward respectively. Lateral cursor movement would later equate to turning the power wheelchair. Using the computer interface, participants performed simulated center-out reaching, typed sentences on a virtual keyboard and played various games, from ping pong to solitaire. For simulated reaching, users performed 24 center out reaches by moving a small cursor from a central target to eight equally spaced targets as quickly and as accurately as possible. Subject typed a pangram English sentence by hovering over the correct keys on a virtual keyboard. For the remainder of the games, users controlled the location of the computer mouse. A click was activated when the cursor was held in a static location for 500 milliseconds. These tasks involved a combination of accuracy training as well as timing training and allowed users to fully learn the relationship between shoulder movements and changes in the control vector.

Figure 2.

Experimental design. The number on the left is the number of sessions that each group spent completing the tasks on the right. Sessions spanned 1 – 1.5 hours. Reaching, typing and real driving were performed to completion while subjects had 15 minutes each for playing games, and virtual driving.

For safety reasons, participants practiced using the BMI to control a virtual wheelchair prior to driving the real wheelchair. Previous research has shown a positive effect of using virtual reality to train and evaluate power wheelchair driving performance [35]–[37]. Using the virtual wheelchair allowed users to have some practice using the BMI to control a power wheelchair without the risk of injury. The virtual wheelchair used in this study was a modified version of the McGill Wheelchair simulator [38], which uses a commercial grade 3D gaming engine (Unreal Development Kit, Epic Games, USA) to provide a realistic first person perspective view of driving a power wheelchair. The system described in [38] was adapted to use the output from the BMI to control the virtual wheelchair. Participants were exposed to two different environments while using the wheelchair simulator. First, they spent 5 minutes freely exploring an environment that mirrored a floor plan at the Rehabilitation Institute of Chicago. This allowed participants to explore different maneuvers in a completely unconstrained manner. Following free exploration, participants were given 10 minutes to complete an obstacle course in a second virtual environment. This course required them to practice an array of maneuvers that mirrored the skills necessary to achieve safe and accurate real world driving. The maneuvers were also similar to the maneuvers that participants would be tested on when driving the real power wheelchair. Participants S1, S2, and S3 were introduced to the virtual driving environment on their 17^th training session, allowing them to practice driving the power wheelchair in the virtual environment for 8 sessions before driving the actual power wheelchair. The number of training sessions was determined from a previous experiment [29] and augmented to ensure that subjects reached steady state performance using the computer interface before driving the real wheelchair.

After 24 practice sessions, subjects S1, S2, and S3 performed 5 sessions driving a power wheelchair using our BMI. Each session lasted about 1.5 hour. During driving sessions, subjects first performed one set of computer reaching followed by 15 minutes of virtual driving. Participants then performed a set of actual wheelchair maneuvers using first their personal joystick, and then the same maneuvers using the BMI. The set of maneuvers was a subset of the wheelchair skills test [39]. This test includes a comprehensive set of tasks encompassing the maneuvers that power wheelchairs users need to be able to make to effectively navigate in real world situations. The nine specific maneuvers that were tested for this study included driving straight forward, driving straight backwards, turning left and right while driving forward and backward, navigating a four cone slalom twice, and driving through a simulated doorway (Figure 2B). The simulated doorway consisted of two traffic cones placed 1 meter apart, through which the participant needed to drive. All participants completed the real wheelchair driving using their personal wheelchair. The maximum forward speed of the wheelchair was held below 1 mph at all times during testing. The three participants used their personal wheelchairs so the maximum angular rate could not be easily set equal for all, however it was kept appropriately low to ensure safety.

A similar experiment was conducted involving the three unimpaired control participants. The only difference between the two protocols was the number of training sessions. Control participants participated in five training sessions before completing one driving session with the power wheelchair. Additionally, control participants were exposed to the virtual reality environment after only three training sessions and practiced in the virtual environment twice before driving the power wheelchair. The length of each training session, the tasks completed during training, as well as the driving maneuvers were identical between groups. The experimental design can be seen in Figure 2. Control participants used a Quantum Q6 Edge power wheelchair (Pride Mobility Products Corp., Exeter, PA) to complete the real wheelchair driving.

Data Analysis

Driving performance was quantified by three metrics. For each maneuver, the path length traversed, the time to completion, and the smoothness of the maneuver were recorded. The smoothness was evaluated by the number of independent sub-movements that the users make in order to realize the complete maneuver and was calculated as the inverse of the number of peaks in the velocity profile. For a given maneuver, peaks were taken only when the velocity was greater than 25% of the maximum velocity and at least 500 milliseconds apart. Prior to peak detection, the velocity profile was smoothed using a second order Butterworth filter with a cutoff frequency of 10Hz to eliminate high frequency noise. The ability to “blend” and/or reduce in number these sub-movements, thus maximizing the smoothness, is a good indicator of the ability to safely and effectively drive a power wheelchair [38]. For visualization, the value of each performance metric achieved when using the BMI was normalized by the value of the same performance metric achieved using the joystick for each maneuver. From this, a performance metric equal to 1 indicates that the participant was able to achieve performance values using the BMI that were equal to the performance values achieved when using the joystick. For completion time and path length, a value greater than 1 indicated that performance using the BMI was worse than performance using the joystick. The opposite was true for smoothness. To quantify final performance using the BMI, a paired sample t-test was performed for each participant for each performance metric between the value achieved using the BMI (9 maneuvers) and the value achieved using the joystick (9 maneuvers) only during the final driving session. To quantify learning, a paired sample t-test was performed for each subject on each performance metric (9 measures per subject) between BMI performance during the first driving session and BMI performance during the fifth driving session. Non-normalized data was used for significance testing. For all statistical tests, a significance threshold of P = 0.05 was used. The average distance that the cursor traveled during the two slaloms was measured for each subjects and each session. Qualitative comparisons were made between the average cursor distance in the first session and the fifth session.

To analyze the dimensionality of the movements, we performed PCA on the raw sensor data taken during different tasks. Specifically, the planarity of movements was calculated as the percentage of variance accounted for by the first two principal components. The planarity was assessed for both slalom maneuvers and comparisons were made between the planarity during the first session and the planarity during the fifth session. Additionally, the body vector can be decomposed into two orthogonal components (h_T and h_N) where A * h_T = p and A * h_N = 0. In this scenario, h_T is in the task space of A and h_N is in the null space of A. This decomposition is further described in [40] and is analogous to the decomposition in [41]. From this, we measured the relative amount of task space variability and null space variability during the two slaloms during the first and fifth driving sessions. The estimated inverse map used during driving (B̂_drive) as well as the estimated inverse map used during reaching (B̂_reach) were also compared for each of the five driving sessions. The inverse maps were calculated according to equation 3. To quantify the difference between the control strategies, the L² matrix norm of the difference between the two inverse maps was calculated for each session. To detect whether these differences were significant or could be attributed to noise, we compared them to the mean of the norm of the difference between the estimated inverse maps used during subsequent driving sessions across all subjects (noise level). If the difference between B̂_drive and B̂_reach was above this noise level, it would suggest that participants were not utilizing the same inverse map, or control strategy for reaching and driving.

Results

Wheelchair Control

All participants were able to control the powered wheelchair with the BMI. Their performance was comparable with the performance with the joystick in terms of distance traveled. As for timing and smoothness, subjects took generally longer to complete the navigation and were less smooth using the BMI, although the performance improved across the experiment sessions.

Figure 3A shows the normalized path length ratios for the three SCI participants. Figure 4 also shows a representative wheelchair trajectory using the BMI (grey) and the joystick (black). All non-normalized path lengths fell between 3 meters and 12 meters. For example, it took S1 4.0 meters to make a right turn using the BMI and 4.4 meters to make the same maneuver using the joystick. A paired sample t-test for each subject revealed that only the path length for S2 when using the BMI was significantly different from the path length achieved when using the joystick (S1: p = 0.735, S2: p = 0.002, S3: p = 0.719) during the first driving session. However, for S2, the path length significantly decreased from the first to the fifth driving session (p = 0.016) and there was no significant difference between the path length using the BMI and joystick during the final session (p = 0.125). The mean normalized time to completion during the first and final sessions can be seen in Figure 3B. The range of non-normalized completion times was 20 seconds to 2 minutes. As a representative case, S1 took 40 seconds to make a right turn using the BMI and 28 seconds to make the same maneuver using a joystick during the final training session. During the first driving session, S1, S2, and S3 took 1.99 (STD = 0.468), 2.35 (STD = 0.572), and 2.04 (STD = 0.477) times longer respectively to complete the maneuver when using the BMI compared to the joystick. All three subjects, however, showed a significant decrease in time to completion (paired t-test p < 0.001, p = 0.015, p = 0.029) from the first driving session to the last. Finally, the mean normalized smoothness can be seen in Figure 3C. In the first session, S2 and S3 were significantly less smooth when using the BMI compared to when using the joystick. Despite this, both subjects S2 and S3 showed a significant increase in smoothness (p = 0.015 and p = 0.029). During the first session, S1 achieved smoothness values using the BMI that were not significantly different from the values achieved using the joystick and thus did not show a significant increase in smoothness over driving sessions (p = 0.326). During the final driving session, no subjects showed a significant difference between the smoothness when using the BMI and joystick (p = 0.306, p = 0.095, p = 0.091).

Figure 3.

Driving performance across of sessions. The three plots show each of the three performance metrics, path length (A), completion time (B), and smoothness (C). All measures were normalized by joystick performance. Different colored lines represent different subjects. Error bars represent 95% confidence intervals.

Figure 4.

Representative power wheelchair trajectories. The wheelchair trajectories for the slalom maneuver are shown for S1 using the BMI (grey) and the joystick (black). There was no significant difference between the two path lengths.

Figure 5 shows the same driving performance metrics for the control participants alongside the SCI participants. The control participants practiced with the interface for five sessions before driving. Control participants achieved slightly worse but comparable normalized smoothness values compared to SCI participants: 0.88 (STD = 0.16) for controls and 0.75 (STD = 0.14) for SCI participants. Control subjects were 2.34 (STD = 0.59) times slower using the BMI compared to the joystick, comparable to the relative time to completion for the SCI participants. Despite being slower and less smooth using the BMI compared to the joystick, no control participants had any problems completing all of the maneuvers and all participants were able to achieve all of the maneuvers without making erroneous movements or colliding with the cones.

Figure 5.

Performance metrics for SCI and control participants. The three performance metrics are shown for the SCI participants (black, left) and the control participants (grey, right). For both the SCI and control participants, the data are from the first driving session after completing computer training. Error bars represent 95% confidence intervals.

Cursor Control

The average distance that the cursor moved during the two slaloms during the first and fifth session can be seen in Figure 6. Over five training sessions, all subjects decreased the average cursor distance for both slaloms. S1 went from 1.65 meters in the first session to 1.17 meters in the fifth session. S2 went from 2.56 meters to 2.30 meters and S3 went from 3.38 meters to 1.98 meters. This equates to an average decrease in cursor path length of 26.8%.

Figure 6.

Cursor distance as a function of session. The mean cursor distance (in pixels) for the two slaloms was measured during the first the final driving sessions. Different colored lines reflect different subjects. Error bars represent 95% confidence intervals.

Dimensional Analysis

In addition to driving performance, we also considered how subjects learned to reorganize their body motions, as captured by the sensor signals. All subjects learned with practice to reduce the amount of motion that was not essential to the performance of the task and they appeared to develop a single control strategy for performing different tasks, i.e. for navigation control and for performing reaching movements with the computer cursor.

Figure 7 shows the change in variance accounted for (VAF) by the first two principal components during the slalom maneuver as well as the ratio of task space variability to null space variability. All subjects increased the VAF by the first two principal components from the first session to the final session. However, the extent of this trend was not equal for the three subjects. Two of three participants (S1 and S3) showed a noticeable increase in the VAF for the first two principal components from the first session to the final session, while one participant (S2) did not show a substantial increase. Similarly, S1 and S3 also showed an increase in the task space variability to null space variability ratio while S2 did not change. Analysis of the inverse map used for driving and reaching is summarized in Figure 8. The figure shows the mean noise level ± 1 standard deviate (grey box) along with the difference between the inverse map used during driving and the inverse map used during reaching for each subject across all five driving sessions (grey circles). The mean noise level was 0.205 mm/rad (STD = 0.063 mm/rad). This difference was greater than one standard deviation above the mean noise level for only 4 out of 15 cases and was greater than two standard deviations for only 1 case. The mean difference between the two inverse maps across all subjects was 0.237 mm/rad (STD = 0.057 mm/rad). A t-test revealed no significant difference between the difference between the inverse map used for driving and reaching and the noise level (p = 0.206).

Figure 7.

High dimensional driving analysis. The top plot shows the change in variance accounted for (VAF) by first two principal components while driving the slalom for each subject in the first and final sessions. The bottom plot shows the ratio of task space to null space variability.

Figure 8.

The difference in inverse map used for reaching and driving across all sessions. Each circle indicates the norm of the difference between the estimated inverse map used for reaching (B̂_reach) and driving (B̂_drive). Different colors represent different subjects. The grey area represents the noise, and is the mean difference between the estimated inverse for subsequent driving sessions averaged across all subjects ± 1 standard deviation.

Discussion

The goal of this study was to test the efficacy of using a body-machine interface to translate high dimensional shoulder movements into low dimensional control signals. Unlike previous studies that investigated using the BMI to control a computer cursor [25, 26, 29], this study focused on using the BMI for controlling a power wheelchair. We hypothesized that individuals with spinal cord injuries would be able to use the BMI to control a power wheelchair, after a moderate amount of training. Additionally, we expected that users would be able to transfer their learned ability to use the BMI for controlling a computer interface to the use of the BMI for controlling a power wheelchair, despite large differences between the tasks. Results suggest two main findings. First, participants with high level spinal cord injuries were in fact able to use small shoulder movements to safely and accurately control a power wheelchair. Second, we found that participants did need some practice driving the wheelchair to learn the inverse relationship between wheelchair movement and cursor movement, however, they still used similar strategies to control the cursor for reaching and driving.

In this study we considered joystick use as a gold standard for power wheelchair control. We did not expect to surpass manual dexterity, among those in whom this was available, and did not expect to supplant the control interface with which participants have many years’ experience. Despite these disadvantages facing the BMI, we wanted to compare our system to the highest possible standard, joystick control. Previous results indicate that individuals with no arm function are able to control the computer interface equally well [31], suggesting that individuals with no arm function would be equally skilled in using the BMI to drive a power wheelchair. Furthermore, we also consider use of the BMI as a way to promote upper body health and mobility in people with joystick control capabilities. Results indicate that participants were able to control a power wheelchair using the BMI at a level that was only slightly worse than when using a joystick. The ability to achieve path lengths using the BMI that were equal to those achieved using the joystick shows that subjects did not make mistaken or unnecessary movements when using the BMI. In fact, all subjects were able to realize their desired trajectory without veering off course or colliding with any cones. More surprisingly, after only a few driving sessions, subjects were able to achieve wheelchair trajectories using the BMI that were nearly as smooth compared to the trajectories achieved using the joystick. The only participant that did not significantly improve performance (S1), exhibited equal smoothness between the BMI and joystick during the first session, which likely caused a ceiling effect. Smoothness is a good measure of how well users are able to control power wheelchairs and suggests that they were using the same general strategy to control the power wheelchair when using both control interfaces [42]. This is especially promising considering that the participants with spinal cord injuries were all expert power wheelchair users and had at least 2 years of experience using a joystick, and retained ample arm control sufficient to operate a standard joystick.

Participants were, however, significantly slower when using the BMI, even after training. Despite this, all showed a significant decrease in completion time after only five training sessions. The slow completion time can likely be attributed to differences in previous experience with the two control interfaces. Joysticks are ubiquitous in video and computer games and are one of the most popular input devices for manual 2D control. It was therefore not surprising that subjects, even those with no previous power wheelchair experience, were more hesitant when using the BMI, a completely novel control interface with which subjects had little experience. This resulted in not only slower speed but also frequent stops to adjust the heading of the wheelchair (reflected in the initial smoothness). As we expected, the gap between BMI and joystick performance shrunk over practice using the BMI or as users became more comfortable controlling the power wheelchair with upper-body movements.

In the current study, we compared BMI driving performance to performance using a standard joystick, however it is likely that a better comparison would be between BMI driving performance and performance using a sip-and-puff or head array. The targeted population for the proposed control interface consists of individuals with absent or marginal control of arm and hand motion, who currently use alternative controllers. The BMI addresses specific disadvantages of the alternative controllers [43] by having proportional control and increased customizability. While the BMI has technical advantages over discrete alternative controllers, the advantages that the BMI has over a standard joystick is dependent on the user’s motor abilities. The BMI can be used by individuals who may not have enough hand or arm function to control a standard joystick; however, for individuals who have sufficient arm control, it is unlikely that the BMI will outperform a standard joystick. In the current study, all participants with spinal cord injuries retained sufficient arm control to safely and effectively use a hand controlled joystick. Further work is needed to compare the proposed interface to commercially available alternative power wheelchair controllers for individuals with severely limited arm and hand motor ability. There are also several practical considerations that must be addressed in the future. Importantly, while no participants reported any abnormal fatigue when using the interface, theoretically, long trips could fatigue shoulder muscles. It would also be advantageous to transition to smaller, wireless, and more cost effective IMUs. We do not feel, however, that these considerations take away from the main findings of this work.

Looking solely at the use of the BMI, it is interesting to note that participants become more efficient at driving the power wheelchair through only a few practice sessions. The results in Figure 7 show that two of three subjects increased the planarity of shoulder movements through driving practice. The other subject (S2) likely experienced a ceiling effect as the first two principal components accounted for 92% of the variance during the first session. Interestingly, the two subjects who increased the VAF by the first two principal components also increased the ratio of task-space variability to null-space variability. This suggests that these subjects’ shoulder movements were not only becoming more planar but their movements were becoming more efficient in the control space. Because the movements of S2 were essentially planar at the start of driving, this subject likely had a very strong mastery of a specific control strategy and thus did not make any adjustments when introduced to the power wheelchair. In addition, all subjects showed a significant decrease in the path length of the cursor while not changing the path of the power wheelchair during the two slaloms. Thus, a measure of control efficiency, or the ratio of cursor movement to wheelchair movement, increased through training for all participants. Overall, this suggests that while subjects had mastered the computer interface prior to driving the power wheelchair as demonstrated by their ability to successfully complete all maneuvers during the first driving session, some practice was needed to learn the most efficient relationship between cursor movement and wheelchair movement.

The virtual environment that we used in this study was not fully immersive. Therefore, it may not be surprising that after practicing the driving of the virtual wheelchair, additional experience driving the real power wheelchair was needed [36, 42, 44]. Despite this, the current study revealed that participants utilized the same inverse map when controlling the computer interface and the power wheelchair. This validates the hypothesis that training with the computer interface is useful for learning to drive a power wheelchair. While the ability to transfer learned skills from a virtual environment to a real world scenario has been established for skills such as driving a car or flying a plane [45, 46], it had yet to be investigated for a redundant control interface with infinite possible control strategies. The fact that users conserve control strategies when switching between cursor control and wheelchair control is especially important in that it is critical to have a training paradigm for a novel wheelchair control interfaces that minimizes risk. The unimpaired participants yielded similar results and helps validate that the proposed control scheme could be useful for individuals with a wide range of abilities.

While here we compared the performance with the BMI to the performance with the joystick controller, we must also stress that engaging the residual upper-body motions in the control task has some important collateral features, beside the ability to drive a power wheelchair. The BMI is also and perhaps more importantly a means to keep the body engaged in performing coordinated motor control tasks. Unlike the brain-machine interface and joystick controllers, the body machine interface can also be programmed to promote physical exercise and to challenge the users to engage parts of the body that lie on the boundary of the paralysis or that tend to be underused [47]. This benefit is critically important to promote recovery and prevent comorbidities in severe paralyses and can be obtained by programming the body-machine map and/or by placing the IMU sensors so as to target specific degrees of freedom of the user's body [48]. Further studies will be conducted to evaluate the efficacy of our BMI system to serve as a rehabilitative tool for individuals with high-level SCIs.

Conclusion

This study investigated the efficacy of using a BMI to control a power wheelchair for individuals with tetraplegia. Shoulder kinematics were measured using inertial measurement units and were mapped to wheelchair control commands using principal component analysis. After training by using the BMI for controlling a computer interface to complete various tasks, participants completed five driving sessions where they performed a number of power wheelchair maneuvers using a joystick as well as the BMI. All participants were able to achieve paths that were not significantly different when using the BMI and the joystick and, after five driving sessions, were able to achieve similar levels of smoothness. Despite being slower when using the BMI, subjects showed a significant improvement in completion time over the driving sessions. Participants were also able to increase their movement efficiency through practice driving. Finally, participants were found to employ similar control strategies when using the BMI for controlling the computer and driving the wheelchair. Overall, the results suggest that the BMI can be an effective power wheelchair control interface for individuals with high-level spinal cord injuries.