Development of an EMG-controlled Serious Game for Rehabilitation

Abstract

A majority of the 7 million stroke survivors in the U.S. have hand impairments, adversely affecting performance of a variety of activities of daily living, because of the fundamental role of the hand in performing functional tasks. Disability in stroke survivors is largely attributable to damaged neuronal pathways, which result in inappropriate activation of muscles, a condition prevalent in distal upper extremity muscles following stroke. While conventional rehabilitation methods focus on the amplification of existing muscle activation, the effectiveness of therapy targeting the reorganization of pathological activation patterns is often unexplored. To encourage modulation of activation level and exploration of the activation workspace, we developed a novel platform for playing a serious game through electromyographic control. This system was evaluated by a group of neurologically intact subjects over multiple sessions held on different days. Subjects were assigned to one of two groups, training either with their non-dominant hand only (Unilateral) or with both hands (Bilateral). Both groups of subjects displayed improved performance in controlling the cursor with their non-dominant hand, with retention from one session to the next. The system hold promise for rehabilitation of control of muscle activation patterns.

I. Introduction

Much of the motor impairment seen after stroke results from aberrant muscle activation. Difficulties range from trouble with fully activating muscles [1] to deactivating them once excited [2]. Excessive coactivation of muscle pairs or groups commonly results in unintended movement direction e.g., hand closing when trying to open [3]. This coactivation can be exacerbated by repeated attempts to perform a task [4], or by simultaneous activation of other muscle groups such as in the proximal arm [5] or the legs [6]. Furthermore, stroke survivors often exhibit reduced capacity to modulate the level of activation of a given muscle with the task [7] and diminished capability to create distinct activation patterns appropriate for specific tasks [8]. In essence, if one considers the space formed by all potential activation patterns of a group of muscles, the region of that space actually accessible by stroke survivors is drastically reduced [9]. This highlights a critical need for rehabilitation strategies targeted directly at improving the repertoire of muscle coordination.

Efforts to directly address muscle activation patterns for rehabilitative training, however, have been relatively rare. Unlike motion or force, they are difficult to sense in the clinic. While a number of researchers have employed electromyographically (EMG) triggered neuromuscular electrical stimulation [10–12], these approaches typically emphasize the amplification of existing muscle activation patterns rather than attempting to improve movement repertoire by reorganizing existing activation patterns or eliciting novel ones. In fact, typical approaches for EMG-triggered therapy use signals from the non-paretic limb [13] to initiate electrical stimulation (or to drive an exoskeleton), rather than employing signals from the impaired limb.

One method for directly training muscle activation patterns entails mapping them directly to a readily visible output, such as a cursor on a computer screen. This has been done for body-machine interface techniques, including for explorations of motor control [14, 15]. One study used EMG control of a cursor to train reduction in muscle coactivation after stroke [16]. Activations of individual muscles were mapped to cursor movement on the computer screen. For stroke survivors, biceps activation controlled cursor movement in the x-direction and anterior deltoid contraction controlled movement in the y-direction. Training consisted of creating independent activation of each muscle to move the cursor to one of two targets. As a result, participants learned to reduce coactivation of these muscles. While this technique showed promise, training was limited to two muscles, each in isolation, and subjects noted a desire for a more engaging interface.

To address these issues, we developed a new platform for therapy targeting muscle activation patterns. As we were interested in patterns involving multiple muscles concurrently, we created a mapping scheme involving principal components to project the n-dimensional EMG space (up to 8 muscles) onto the two-dimensional space of the screen. Thus, the user can manipulate cursor location by generating activation patterns defined across multiple muscles. By controlling the mapping between activation patterns and cursor movement, we aim to guide exploration of the activation workspace. To address issues with engagement and encourage exploration of the activation workspace, we created custom exercises, or serious games. Additionally, to facilitate use in those with greater impairment, we developed both a unilateral mode, in which EMG signals from one arm drive the cursor, and a bilateral mode in which a weighted sum of signals from both arms control the cursor location.

To assess system performance, we conducted a pilot study with neurologically intact individuals, half of whom used the bilateral mode and half of whom used the unilateral mode. Based on prior work on learning in body-machine interfaces [17, 18], we hypothesized that EMG control of the cursor would improve across multiple sessions with continued practice. We further expected that users would alter their preferred activation patterns to better align with the mapping between EMG and cursor position.

II. System Design

A. System Development

The intent of the system is to encourage exploration of the muscle activation workspace. The user must manipulate EMG signals to control the position of a cursor on the computer screen in order to play serious games.

B. Selecting Target Vectors

Two EMG patterns are mapped to the x-and y-axes, respectively, of the computer screen. These target vectors may be selected in a variety of ways. First, they can be entered directly into a custom graphical user interface (GUI) created using the MATLAB GUI environment. Second, the vectors can be read from a file specified through the GUI. Third, to shape the tasks to the capabilities of the user, they can be derived from the user’s current activation workspace. For this third approach, the GUI cues the user to perform a brief (1-minute) initial calibration phase in which a variety of muscle activation patterns are created to span the voluntary workspace (Fig. 1). Principal components (PCs) are calculated from the eigenvectors of the covariance matrix for this dataset. From the GUI, the user or therapist is able to select which two PCs will serve as the target vectors, aligned with the two axes of the computer screen. The choice of PCs can be used to encourage exploration of the workspace and to maintain an appropriate level of challenge (e.g., increasing challenge by starting with the two PCs which explain most of the variance in the EMG dataset and then progressing to PCs which explain less of the variance). Additionally, the PCs for one limb can be used as the target PCs for the other limb, as might be advantageous for stroke therapy.

Fig. 1. Calibration gestures.

Examples of some of the hand gestures subjects were cued to perform in order to encourage exploration of the muscle activation workspace.

Once the target vectors (hereafter generically referred to as PCs) are chosen, they are then used to select the region of the muscle activation space to be explored. The two selected PCs define a hyperplane within the activation workspace. This region can be represented in two-space as movement along a PC_i-axis and a PC_j-axis. The area within this PC_i-PC_j plane to be used for therapeutic practice can then be demarcated through a custom GUI (Fig. 2). On the GUI, the therapist selects the desired activation range for each muscle (maximum range of 0%−100%). This forms a hypercube of the allowable activation space. This hypercube is then projected onto the PC-PC plane, thereby creating a polygon within this plane. The therapist then defines the portion of the PC-PC plane to be used by drawing a rectangle within the polygon (Fig. 2). The larger the side of the rectangle, the greater the activation range of the corresponding PC needed to move the computer cursor fully across the screen.

Fig. 2. Graphical user interface for selecting target vectors.

Radio buttons to the right allow selection of target vectors for mapping EMG to the computer screen, either from PCs or from a file or the keyboard. Middle graphs display the PCs computed from the calibration data for the dominant (top) and non-dominant (bottom) hands. Left graphs display the achieved exploration (magenta) within the projection of the hyperplane defined by the selection of the chosen PCs. The polygon (blue outline) shows the projection of the hypercube representing the allowable activation range of each muscle. Through the GUI, one can size the rectangle (red outline) to select the area of the PC-PC plane to be mapped to the computer screen. Slider bar in lower right corner controls relative weighting of EMG signals for bilateral control.

C. Mapping EMG to Cursor Location

Muscle activation is estimated in real-time from filtered EMG signals. At each time point, the vector of normalized EMG activations (each component ranging between 0 and 1) is projected onto the hyperplane formed by the two orthogonal target PC vectors. The location within the hyperplane is then mapped to the corresponding location on the computer screen and the cursor is moved to this position.

For bilateral use, EMG signals in both limbs are used to control a single cursor. A variable relative weighting is applied to the EMG vectors from each extremity to compute the final vector, ultimately used to control the cursor. For example, initially 100% of the weight could be applied to one limb and then gradually over time (either within a session or over several sessions) the weight could be shifted toward the other limb. The weighting is easily adjusted through a slider bar located on the GUI (Fig. 2). This training paradigm could be helpful for stroke therapy, wherein initial control of the cursor is dominated by the less impaired arm; as performance improves, cursor control gradually shifts to the more impaired limb.

D. Training Environments

Different training environments were created in Simulink to encourage exploration of the activation workspace. The underlying structure for most of these exercises is a square grid, formed by segmenting the computer screen into an n × n array of square tiles (default for most exercises is 5 × 5). If more or fewer elements are desired, the therapist can change the dimension of the array through the main GUI, which controls selection of exercise type and parameters.

The user manipulates his or her EMG patterns to move the cursor around the tiles. The outer border of the tiles changes color from its default white to cue the user when muscle activation patterns are being created that fall outside of the specified ranges. Whenever one or more muscles is or are activated beyond the range specified (see Selecting Target Vectors), all of the borders turn magenta, indicating that the subjects should relax their muscles. Alternatively, if muscle activation level falls below the minimum threshold, the borders all turn yellow to prompt the user to increase muscle activation. Should the user produce an EMG vector that would drive the cursor outside the array of tiles on the screen, the crossed border turns red.

Four serious games, or exercises, were created to increase engagement and guide exploration. The Target exercise involves moving the cursor to the specified tile by providing an appropriate EMG pattern (Fig. 3a). The target tile turns green to indicate that the movement should begin. The visual display provides continuous feedback of the current location of the cursor in relation to the location of each tile. Once the user successfully moves the cursor to the targeted tile and keeps the cursor “stationary” for a prescribed amount of time, the tile turns yellow. The stationarity criteria require that the cursor move less than 40% of the tile width for at least 9 out of 12 consecutive frames, or 0.9 sec. A new target tile, not previously targeted, is then randomly chosen; it turns green to signal the start of the next movement. The exercise continues until all the tiles have been targeted once.

Fig. 3. Serious games.

A) Target exercise. User controls cursor location (red dot) to move to the indicated square (green with cross). B) Picture Reveal exercise. Moving the cursor into the white square causes the square to disappear, revealing part of the hidden picture. C) In Targeted Picture Reveal exercise, the user must erase each square in an indicated order (indicated by the active square turning green). D) In Maze exercise, the user enters the maze by moving the cursor which moves freely in the game board to the start square. After that the walls will become active and the user should move the cursor through the maze to the finish square. E) In the Coin Collector exercise, the user tries to gather the coins while avoiding the asteroids.

Picture Reveal is a related exercise for which the user also attempts to move the cursor to different tiles (Fig. 3b). For this exercise, once the cursor meets the stationary criteria, the tile disappears to reveal part of a picture beneath. By moving to different tiles, the user is able to reveal more of the picture and is free to choose the order of the revealed tiles. Targeted Picture Reveal increases the level of difficult by forcing the user to reveal the tiles in a specified order (cued by the next tile turning green) chosen at random by the computer (Fig. 3c).

In the Maze exercise, based upon a game written in MATLAB by Rodney Meyer [19], the user must control the cursor in order to find a path through the maze (Fig. 3d). The cursor cannot move through the walls of the maze, so the user must modify their activation patterns to maneuver the cursor around the walls. Visual feedback of the path chosen is provided by marking the current tile location of the cursor with a red diamond and previously occupied tiles with yellow diamonds. To decrease the difficulty of the exercise, we implemented an option to prevent the cursor from inadvertently losing progress. With this option, a wall is introduced after every correct step of the cursor to prevent movement away from the target. As the user improves, this bumper option can be removed. Additionally, a different maze with a unique solution path is created each time the exercise is called.

The goal of the Coin Collector exercise, based on the Asteroids game written by Héctor Corte in MATLAB [20], is to collect stationary coins which appear on the screen by moving the cursor (represented as a spaceship) to the coin without striking moving asteroids (Fig. 3e). Points are awarded for each coin collected. Challenge is maintained by increasing the number and average speed of asteroids as the user collects more coins.

To increase the accessibility of the game for users with visual issues, the cursor color and size are also adjustable through the main GUI. Additionally, the GUI provides feedback to the therapist, by displaying the muscle activations being produced by the user, so that the therapist can guide the user as needed to alter their activation pattern.

E. Evaluations

All data from each session can be recorded. For example, system parameters can be saved to a file specified through the main GUI for future analysis or use in subsequent sessions. Duration of use, the exercise performed, specified exercise options, activation to computer screen mapping, and criteria of game play can be stored. These data provide information about system usage, difficulty of the game, and progress of the subject. Furthermore, all EMG data are recorded throughout each exercise for future analysis.

We included a test task in the system to assess EMG control of the cursor. This test task is a version of the Target exercise in which the computer screen is divided into a 5 × 5 set of tiles. A total of four of these tiles, representing some of the most challenging movements, are used as targets: upper left, upper right, center, and bottom center (Fig. 4). For the test, each of the four locations must be reached four times in a specified order. Target order for the 16 total targets is randomized. The system records the total time required to reach all targets.

Fig. 4. Targets in the Test evaluation.

Target locations in the game board for the Test evaluation of cursor control. In the actual Test, only one target is illuminated at a time.

III. Pilot Study

A. Protocol

To assess system performance, we conducted a pilot study with healthy, neurologically intact adults. A total of 20 participants (7 Female/13 Male) were enrolled after providing informed consent in accordance with procedures specified by the Northwestern University Institutional Review Board, which oversaw the study. Mean age was 27 ± 5 years and ranged from 21–40 years. Participants were randomly assigned to one of two groups.

In order to simulate a potential therapy paradigm for stroke survivors, in this study the first two PCs from the dominant arm were used as the target vectors for the non-dominant arm. This mimics a scenario in which the activation patterns in the non-paretic arm are used as the target patterns for the paretic arm for stroke survivors

The Unilateral group used only their non-dominant arm to control the cursor location on the screen during the training game play. The Bilateral group used both arms to control the cursor. Cursor position was computed as a weighted sum of the cursor positions defined by the activation patterns in each limb:

$$CP=W_{DH}\ast C{P}_{DH}+\left(1-W_{DH}\right)\ast C{P}_{NH}$$ (1)

where CP is cursor position, W_DH is the weighting coefficient of the dominant hand (ranging from 0 to 1), CP_DH is the cursor position calculated from the dominant hand EMG, and CP_NH is the cursor position calculated from non-dominant hand EMG.

The weighting coefficient W_DH was gradually adjusted by a member of the research team from full control by the dominant hand to full control by the non-dominant hand throughout the course of each session. The transition was gradual and unidirectional. As the user gained confidence in curser control, W_DH was decreased in 5% – 10% steps.

Four wrist and finger muscles were targeted in each limb. Thus, for the Unilateral group, four muscles in the non-dominant limb determined cursor location, while for the Bilateral group four muscles in each limb potentially impacted cursor location. EMG signals were recorded from extensor digitorum communis (EDC), flexor digitorum superficialis (FDS), extensor carpi ulnaris (ECU), and flexor carpi radialis (FCR) muscles in both arms using active, surface electrodes (Bagnoli, Delsys, Inc., Boston, MA). Electrode placement over these superficial muscles was guided by anatomical landmarks and palpation. The EMG signals were fed into the system for processing, as described in the previous session.

EMG signals were sampled at 1 kHz under control of the Simulink DAQ software. The raw EMG signals were bandpass filtered using a minimum order Butterworth filter with stopband frequencies of 20 and 450 Hz, passband frequencies of 35 and 435 Hz, and stopband attenuation of 20 dB. To quantify EMG magnitude, the EMG signals were rectified and low-pass filtered using a minimum order Butterworth filter with bandpass frequency of 1.13 Hz and stopband frequency of 5 Hz. We selected filter parameters to create smooth cursor movement. The EMG envelope was subsequently normalized by the maximum envelope value for the corresponding muscle, as recorded during maximum voluntary contractions (MVCs) performed at the start of each session.

Each of the three training sessions began with a calibration phase. First, EMG signals were captured during MVC in order to normalize the signals. Subjects were asked to create specific gestures using MVC of each of the four muscles of interest. All sampled EMG data were subsequently normalized by the maximum values obtained during MVC contractions.

Next, participants were encouraged to fully explore the muscle activation workspace in order to find the target PCs. Each subject was shown the aforementioned series of gestures and movements (Fig. 1) and was instructed to replicate them with up to 50% of MVC. The calibration EMG vectors, projected onto the plane mapped to the computer screen, were displayed along with the PCs. Calibration was repeated if necessary to ensure good coverage of the calibration plane.

PCs were computed for the EMG data from this Calibration phase to determine the target activation vectors. For both the Bilateral and Unilateral groups, the first two PCs (the ones explaining most of the variance) for the dominant hand data were employed as the target vectors mapped to the axes of the computer screen.

After completing the Calibration phase, the participant performed the Test with the system (Fig. 4). The testing was followed by a Training phase in which the participant played a combination of Target and Picture Reveal exercises with the system. At the end of 30–45 minutes of game playing, the subject completed another Test. For both groups, all Tests were performed with the non-dominant hand only.

B. Analysis

Data from each of the 6 Tests performed over the three sessions were analyzed to examine the impact of training and group on test performance. Time to completion, EMG patterns employed, and cursor kinematics recorded during these phases were analyzed.

First, we wanted to examine whether practice with the system led to improved task performance. The time to complete each of the 6 Testing phases was assessed using a 3-way (Session × Practice × Group) repeated measures analysis of variance (rmANOVA). The within-subject factors were Session (Day 1, Day 2, Day 3) and Practice (Pre-training, Post-training). The between-subject factor was training Group (Unilateral, Bilateral). The Greenhouse-Geisser correction was used when sphericity assumptions about the variance were violated.

Second, in order to examine potential changes in activation patterns produced by training, we examined EMG signals during the Test. We computed PCs for the EMG signals recorded from the non-dominant arm during each Test. The amount of variance accounted for (VAF) by the first two PCs was computed in order to determine whether these patterns were used more frequently after training. We conducted rmANOVA to examine any effects of Session, Practice, or Group. Further-more, we compared the target hyperplane formed by the PCs of the dominant hand to the hyperplane created by the PCs from the non-dominant hand by computing the dihedral angle between them. We performed rmANOVA to examine potential effects of Session, Practice, or Group on this dihedral angle.

Finally, we evaluated control of cursor kinematics by computing the mean-squared jerk of cursor movement and the path length of the cursor between targets during each Test. Mean-squared jerk was computed through repeated differentiation and filtering of the original position data [21]. The lowpass filter had the same characteristic as the one used to calculate the activation signal. We computed actual path length between consecutive targets from the cursor displacement and then normalized this value by the Euclidean distance between the target locations (minimum path length is equal to 1). The path lengths were summed across all 16 targets to create a metric for path deviation. We employed rmANOVA to test for effects of Session, Practice, or Group on each of these outcome measures.

C. Results

All subjects participated in all three training sessions, held over a period of 3–14 days for each subject. Three subjects had incomplete data sets for one or more of the Tests, so their data were excluded from analyses. Data from two other subjects were excluded due to excessive noise in the EMG signals that precluded proper assessment. Data from the remaining 15 subjects (8 in the Unilateral group and 7 in the Bilateral group) were included in the statistical analyses. Sample cursor trajectories for Day 1 pre-training and Day 3 post-training Tests are shown in Fig. 5. Participants substantially improved control of the cursor, which in turn led to reductions in the time needed to complete the Test. ANOVA results revealed that both Practice (Pre-training, Post training) and Session (Day 1, Day 2, Day 3) had significant effects on completion time (F(1,13) = 62.730, p < 0.001 and F(2,26) = 8.358, p = 0.002, respectively) across the two groups. Completion time decreased by a mean of 67% from the first trial on Day 1 to the last trial on Day 3 (Fig. 6).

Fig. 5. Test game cursor trajectories for a single subject.

Blue trajectories represent the projected path of the cursor based on the activation patterns produced by a subject performing the Test evaluation shown in Fig. 4. A) Day 1, Pre-Training and B) Day 3, Post-training. Red square (5 blocks × 5 blocks) indicates the extent of the computer screen (note that visual representation of cursor did not leave the screen during Test) while Xs indicate target locations. First column shows that trajectories traverse far outside the red square for the first Test game but stay largely within the square for the last Test game. The magnified view in the fight column shows that the subject was able to move more directly toward the targets during the final Test session.

Fig. 6. Testing phase completion time.

Time required to complete the randomized, targeted Test sequence of 16 tiles before (Pre) and after training (Post) on each of the three sessions. Across both groups, completion time decreased significantly for both Session and Practice. There was no significant effect of Group.

There were no significant differences in time to completion, however, between subject groups (F(1,13) = 2.598, p = 0.131). Both groups were able to manipulate muscle activation in the non-dominant limb to control the cursor despite the fact that the target activation plane was defined by the PCs from the dominant limb. The Unilateral group displayed slightly greater improvement on Day 1 (non-significant), but the Bilateral group caught up by Day 2. No interactions were significant.

We anticipated that participants would adapt to favor specific activation patterns that would enhance completing the exercises and with training, more of the variance in EMG data would be explained by the first two PCs. However, neither Session nor Practice had a significant impact on VAF by PC1 (F(2,26) = 0.148, p = 0.863 for Session and F(1,13) = 0.033, p = 0.859 for Practice, see Fig. 7). Furthermore, neither Session nor Practice significantly affected VAF by PC1 and PC2 together (F(2,26) = 0.663, p = 0.523 for Session and F(1,13) = 0.004, p = 0.950 for Practice, Greenhouse-Geisser correction). VAF by PC1 and PC2 actually slightly decreased by an average of 1.3%, from 71.3% to 70.0%, over the three sessions. Subject group (Unilateral vs. Bilateral) did not impact VAF (F(1,13) = 0.212, p > 0.612 for VAF for PC1 and F(1,13) = 0.040, p = 0.845 for VAF for PC1 and PC2). All interactions were not significant.

Fig. 7. Variance accounted for (VAF) by PCs.

Variance in EMG data recorded during the Test that was explained by PC1 (lower part of bar) and PC1 and PC2 together (full bar). No significant differences were seen across Group, Session, or Practice. Error bars represent one standard deviation of VAF for PC1 and PC2 together.

We compared the actual PCs employed by the participants during the Tests with the target PCs by computing the dihedral angle between the hyperplanes formed by the respective sets of PCs (Fig. 8). While these angles decreased slightly from the first to last Tests, changes were not significant. Neither Session (F(2,26) = 1.816, p > 0.183) nor Practice (F(1,13) = 0.014, p > 0.909) affected the angle, which maintained a value of roughly 40°. Group did not affect the angle between the planes, either.

Fig. 8. Dihedral angle between actual and target PCs planes during the Tests.

No significant effects of Session, Practice, or Group were observed.

In contrast, participants exhibited substantially improved cursor kinematics. Total path length decreased by over 75% from the first (920.0) to last (221.3) Test (Fig. 9). Both Practice and Session significantly affected path length (F(1,13) = 30.538, p < 0.001 and F(2,26) = 8.070, p = 0.002, respectively). There were also significant differences between subject groups (F(1,13) = 5.996, p = 0.029). The Unilateral group showed greater improvement in path length across the sessions, although path lengths for the final session were roughly equivalent for the Unilateral and Bilateral groups due to larger path lengths for the Unilateral group on Day 1 (Fig. 9). None of the other interactions terms were significant.

Fig. 9. Path length between targets for the cursor movement during the Tests.

Path length decreased significantly for both Session and Prectice.

Within a given Session, participants also exhibited a reduction in mean-squared jerk of the cursor (Fig. 10). Across groups, Practice significantly affected mean-squared jerk, with a decrease of 9% (F(1,13) = 62.6, p < 0.001). There was a significant interaction between Session and Group (F(2,26) = 4.085, p = 0.029), but no other main effects or interactions had a significant effect on mean-squared jerk (p > 0.066).

Fig. 10. Mean-squared jerk for the cursor movement during the Tests.

With training, subjects in both groups appeared to be able to create smoother movements, as indicated by the reduction in mean-squared jerk.

IV. Discussion

We have developed a system intended to facilitate training of multi-muscle activation patterns in stroke survivors and in other individuals with impairment arising from neurological damage. The system directly addresses primary mechanisms of impairment in stroke survivors, namely, diminished capacity to create unique activation patterns and to appropriately scale activation amplitude with task. Our system affords the flexibility to shape the training to the needs of the individual user through an intuitive graphical interface.

We successfully employed a version of the system with neurologically intact adults in a pilot study intended to examine feasibility. Participants trained with the system for 30–45 minutes on three separate days. In accordance with previous studies involving mapping of hand kinematics [22, 23] or EMG [24] to the cursor location, we observed significant improvement in cursor control both within a session and across sessions for both Unilateral and Bilateral groups. By the end of the first session, for example, subjects reduced the amount of time needed to complete the removal of the 16 targets by over 50%. These gains were largely maintained at the start of the second session, even though the mapping from EMG to cursor location could change with session. Overall, by the end of the third training session, subjects had reduced the time needed for Test completion by 67% (or roughly 281 seconds).

Improvement seemed to arise primarily from better control of existing activation patterns. Path length between targets was substantially reduced (by over 75%) following training. Within a session, mean-squared jerk of the cursor was significantly reduced as well. Subjects training unilaterally further displayed a reduction in mean-squared jerk across sessions. Thus, subjects seemed to improve the ability to control and scale the activation patterns needed to properly move the cursor.

Interestingly, participants did not appear to greatly alter the nature of their preferred EMG pattern, even as control of the cursor improved. The dihedral angle between the hyperplane formed by the target PCs and the hyperplane formed by the PCs from the actual EMG data did not change greatly within a session or across sessions despite substantial separation of the planes (separated by an angle of 40–45° ).

This finding of using existing activation patterns to solve the task rather than reorganizing them is consistent with prior research [25], thereby indicating that the nervous system may have a preference for using habitual coordination patterns. However, these neurologically intact participants undoubtedly improved in their ability to modulate the relative activation of their patterns, as evidenced by the reduced path lengths. Such improvement in modulation would be very beneficial for stroke survivors who struggle to match muscle activation to task [26]. Additionally, stroke survivors may exhibit a greater propensity to shift activation patterns, especially in the acute and subacute phases. Should stroke survivors exhibit a reluctance to explore new patterns, we will add further motivation to change patterns, such as by adding noise to some of the signals [27] or by remapping the task so that the existing coordination patterns no longer result in motion of the cursor [23].

This training methodology holds several appealing features for stroke survivors. First and foremost, it addresses the primary impairment mechanisms for hand function after stroke: creation and modulation of appropriate muscle activation patterns. Fostering greater exploration of the workspace may lead to improved sensorimotor control. In a pilot study focused on fingertip force generation in stroke survivors, we observed that facilitating exploration of the three-dimensional workspace of isometric fingertip force led to improved normal force production for pinch [28]. Second, it encourages practice even when users are incapable of creating intentional finger movements or forces. EMG activation, even if insufficient or inappropriate to perform a physical hand task (like grasping an object), can still produce a functional outcome in the exercises within the interface. This provides the opportunity to reward even small changes in control, and encourage continued practice. Moreover, the difficulty of each exercise can readily be adapted to maintain the appropriate level of challenge [29] for each user.

Finally, the system supports both unilateral and bilateral modes of training. In this study, despite possible advantages for the bilateral mode due to target activation patterns being taken from the dominant limb, equally good improvement was seen in both groups of neurologically intact participants. We envision, however, that stroke survivors with severe hand impairment may benefit from the bilateral training mode, as control of the impaired upper limb may be insufficient to play the games [30]. In this case, contribution from less impaired upper limb would encourage game play and workspace exploration with the impaired upper limb by keeping the challenge at an appropriate level. The contribution from the non-paretic limb can gradually be weaned as the individual improves control of the paretic limb.

For this study, the first two PCs describing activation patterns in the dominant limb were used as target vectors for the non-dominant limb, simulating employment of the activation patterns of the non-paretic limb as targets for the paretic limb after stroke. Alternatively, other PCs (e.g. 3^rd and 4^th) from either limb could be used as target vectors for game play. In addition, while we chose to use PCs to represent the activation patterns, in part due to their orthogonality characteristics, other activation pattern representations (such as non-negative matrix factorization) could be employed. These would, however, require application of an appropriate transformation in order to map such non-orthogonal vectors to the orthogonal x-and y-axes on the screen.

A limitation of our pilot study is that it was conducted with neurologically intact subjects. Therefore, the extent to which the results generalize to stroke survivors is not known. However, as noted, there are several advantages to using this paradigm for stroke rehabilitation. Given that the current approach extends other EMG paradigms that have shown promise [16, 24], we are optimistic that training with this system will be therapeutically effective for stroke survivors. Future studies investigating this potential treatment technique with a patient population should be conducted.