Innovative approaches to the rehabilitation of upper extremity hemiparesis using virtual environments

Abstract

Aim

Upper-extremity interventions for hemiparesis are a challenging aspect of stroke rehabilitation. Purpose of this paper is to report the feasibility of using virtual environments (VEs) in combination with robotics to assist recovery of hand-arm function and to present preliminary data demonstrating the potential of using sensory manipulations in VE to drive activation in targeted neural regions.

Methods

We trained 8 subjects for 8 three hour sessions using a library of complex VE’s integrated with robots, comparing training arm and hand separately to training arm and hand together. Instrumented gloves and hand exoskeleton were used for hand tracking and haptic effects. Haptic Master robotic arm was used for arm tracking and generating three-dimensional haptic VEs. To investigate the use of manipulations in VE to drive neural activations, we created a “virtual mirror” that subjects used while performing a unimanual task. Cortical activation was measured with functional MRI (fMRI) and transcranial magnetic stimulation.

Results

Both groups showed improvement in kinematics and measures of real-world function. The group trained using their arm and hand together showed greater improvement. In a stroke subject, fMRI data suggested virtual mirror feedback could activate the sensorimotor cortex contralateral to the reflected hand (ipsilateral to the moving hand) thus recruiting the lesioned hemisphere.

Conclusion

Gaming simulations interfaced with robotic devices provide a training medium that can modify movement patterns. In addition to showing that our VE therapies can optimize behavioral performance, we show preliminary evidence to support the potential of using specific sensory manipulations to selectively recruit targeted neural circuits.

Improvement in upper extremity post-stroke function has been recalcitrant to current therapeutic interventions with only 5% of all patients regaining full use of their upper extremity following intensive therapy. Stroke rehabilitation has focused on the facilitation of isolated movements through spasticity reduction, passive guidance and graded movement assistance. Current emphasis is on repetitive task practice and utilizing principles of motor learning such as regulating practice schedules and augmenting feedback. It is believed that these motor learning principles parallel the practice principles purported to effect neuroplasticity. Evidence that plasticity is “use-dependent” and intensive massed and repeated practice may be necessary to modify neural organization.^1–3

Virtual reality (VR) technology may be an appropriate means to provide these plasticity-mediated therapies using motor learning principles. The first generation of these computerized systems provided motivating environments in which tasks could be practiced repetitively. Training schedules, specificity and frequency of visual and auditory feedback could be objectively monitored and quantified. Currently treatment interventions are being developed to take advantage of technological advances such as the improvement in robotic design, the development of haptic interfaces, and the integration of these devices with virtual environments. Studies have shown that robotically-facilitated repetitive movement training might be an effective stimulus for normalizing upper extremity motor control in persons with moderate to severe impairments who have difficulties in performing unassisted movements.^4–6 Several authors have integrated VE with adaptive robotic systems to train the hemiparetic upper extremity.^7–10

Although considerable progress has been made in developing these devices, the clinical evidence demonstrating effectiveness of these systems has not yet reached the highest levels of evidence found in systematic reviews and randomized controlled studies. Many of the studies supporting the use of interactive robotics and virtual environments interfaced with movement tracking and sensing glove systems consist of case studies, small feasibility studies, or studies without control groups. Unfortunately, in fact, the assimilation of technology into rehabilitation has not been as fast or as extensive as in other branches of medicine.¹¹

If repetition and skill learning are important for motor learning and recovery of function, what do these technologies add over and above real-world task practice? This is an important question. What can training within an interactive virtual environment, either with or without robotic devices contribute to skill learning and improved motor control? Do gaming environments, augmented visual, auditory and haptic feedback provide added value to the learning process? Are these technological interventions able to model and incorporate accepted rehabilitation practices such as physical assistance and graded progression of tasks to adjust the kinematics of the movement during training? We believe that the combination of virtual environments and robotics can be effectively used both as a training tool and and as a test of hypotheses regarding the benefits of various rehabilitation approaches. These capabilities are afforded through the quantitative evaluation tools inherent to this technology.

We present an example of such a study that tests two different approaches to hand rehabilitation. The prevailing paradigm for upper extremity rehabilitation describes the need to develop proximal control and mobility of the shoulder prior to initiating training on the hand.¹² This has been the accepted rehabilitation method for many years. An increasing number of human and animal studies ^13–16 have reported that movement practice increases the area and density of motor cortex correlated with that movement, with the possibility that this expansion of motor territory influences representations occupying adjacent territory.¹⁶ It is not clear whether this expansion of cortical representations occurs through sharing of cortical tissue among representations ¹⁷ or through competition for cortical territory.^18–20 These findings prompt us to reconsider the rehabilitation strategy that encourages early shoulder activation post-stroke. In general, there is better return of upper arm function post-stroke than of the hand. ²¹ Does early motor activity of the upper arm and shoulder hinder recovery of hand function because of cortical competition facilitated through intensive motor activity? Several small studies exploring the concept of providing additional hand training during conventional therapy ¹⁶ or training the hand while the upper arm is deafferented and deefferented through regional anesthesia ²⁰ have shown positive changes in hand function.

The purpose of this paper is to report preliminary findings on the feasibility of using special designed virtual environments in combination with robotics to assist recovery of hand-arm function post-stroke. The first section presents preliminary findings that address the above mentioned competition hypothesis. Two groups of post-stroke patients were trained: one group used virtual reality training simulations to train the hand and arm together as a functional unit and one group trained hand and arm separately. We hypothesized that balancing and integrating the training of both proximal and distal components of the upper extremity will minimize over-representation of the upper arm.

In the second section, we report preliminary data that demonstrate the potential of sensory manipulations in VE to selectively drive activation in targeted neural regions. For this, we used a task, mirror visual feedback (MVF),²² which in smaller-scale studies has shown promise in aiding recovery of hand-arm function after stroke.^{23, 24} We simulated the MVF effect in VE by creating a “virtual mirror” as subjects performed a unimanual task. We simultaneously measured cortical activation with functional magnetic resonance imaging (fMRI) and, in a separate session, with transcranial magnetic stimulation (TMS). We hypothesized that the virtual mirror effect would be associated with activation of the motor cortex in the hemisphere ipsilateral to the moving hand (i.e. contralateral to the mirror-reflected hand).

Training stroke patients in a virtual environment

Methods

A unique exercise system was developed consisting of complex visual, auditory and haptic simulations that provide for guidance of arm movement in a three-dimensional (3D) space which is adaptive in real time as well as on a trial-by-trial basis. The system consists of interactive virtual reality simulations and external hardware integrated to interact with the virtual environments. A library of gaming simulations was created to exercise the hand and the arm separately and the hand and the arm together.

Hardware

Hand

The system supports the use of Cyber-Gloves instrumented gloves for hand tracking and a CyberGrasp for haptic effects (Immersion Inc., San Jose, CA, USA). The CyberGrasp device is a lightweight, force-reflecting exoskeleton that fits over the CyberGlove (Figure 1B). In this study the CyberGrasp is used to facilitate individual finger movement in patients with more pronounced deficits, by resisting flexion of the adjacent fingers. This allows for individual movement of each finger. The Ascension’s Flock of Birds^® (Ascension Technologies Co., Burlington, VT, USA) is used for arm tracking. Hand position and orientation as well as finger flexion and abduction is recorded in real time and translated into three dimensional movements of the virtual hands shown on the screen in a first-person perspective.

Figure 1.

Human-Robotic Interfaces. A) Hand and arm training system that utilizes a data glove to capture finger movements and a robotic interface that records shoulder, elbow and forearm movements in three-dimensional space, allowing the user to interact with simulations featuring haptically rendered objects; B) hand and arm training system that utilizes a data glove to capture finger movements, and a hand exoskeleton that can train finger flexion strength, inhibit mass grasp patterns or provide haptically rendered collisions. An Ascension’s Flock of Birds^® magnetic tracker was used to track arm movement.

Arm

The arm simulations utilize the Haptic-Master^® (Moog FCS Inc., The Netherlands),²⁵ a 3 degrees of freedom, admittance controlled (force controlled) robot. Three more degrees of freedom (yaw, pitch and roll) can be added to the arm by using a gimbal, with force feedback available only for pronation/supination (roll) (Figure 1A). A three-dimensional force sensor measures the external force exerted by the user on the robot. In addition, the velocity and position of the robot’s endpoint are measured. These variables are used in real time to generate reactive motion based on the properties of the virtual haptic environment in the vicinity of the current location of the robot’s endpoint. This allows the robotic arm to act as an interface between the participants and the virtual environments enabling multiplanar movements against gravity in a 3D workspace. The haptic interface provides the user with a realistic haptic sensation that closely simulates the weight and force found in upper extremity tasks.²⁶

Simulations

We have developed a comprehensive library of gaming simulations to exercise the hand alone, the arm alone, and the hand and arm together. Most of the games have been programmed using C⁺⁺/OpenGL or the Virtools software package with the VRPack plug-in (Dassault Systemes, Suresnes Cedex, France) which communicates with the open source VRPN.²⁷ In addition, two activities were adopted from existing Pong games in which we have transferred the game control from the computer mouse to one of our input devices (e.g., CyberGlove or Haptic Master). We used HapticMaster’s Application Programming Interface (API) to program the robot to produce haptic objects, including walls, blocks, cylinders, toruses and spheres as well as haptic effects, such as springs, dampers and global forces.

Hand simulations

1) The “Piano Trainer” is designed to help improve the ability of subjects to move each finger (fractionation). It consists of a virtual piano, that plays the appropriate notes as they are pressed by virtual fingers (Figure 2A). The simulation can be utilized for training the hand alone (Piano 1) to improve individuated finger movement (fractionation), or the hand and the arm together (Piano 2) to improve arm trajectory as well as finger motion; 2) “Space pong” trains the subjects’ ability to coordinate finger flexion and extension in order to react to and engage a moving target. The participants control the paddle with their finger position. The trajectories of the target are non-predictable, thus necessitating a high level of conscious attention and feed-forward processing. Feedback is provided through the number of successful hits.

Figure 2.

Virtual simulations. A) The virtual “Piano Trainer” presents a complete keyboard that produces realistic, individual notes when pressed by virtual hands which are controlled by the participant’s fingers; B) “Reach Touch” trains point to point reaching movements to haptically rendered virtual targets presented stereoscopically in three-dimensional space; C) “Placing Cups”, trains transport and placing movements shaped by haptically rendered table and shelf; D) “Hummingbird Hunt” requires participants to use a pincer grasp to capture a bird as it flies through a complex visual environment; E) “Hammer task” trains rapid, repetitive, goal-directed elbow extension, shoulder flexion and finger extension against gravity in three dimensions.

Arm simulations

1) “Reach/Touch”. The goal of the Reach/Touch game is to improve speed, smoothness and range of motion of shoulder and elbow movement patterns in the context of aiming/reaching type movements (Figure 2B). Subjects are immersed in a 3-dimensional stereo workspace aided by stereoscopic glasses (RealD/StereoGraphics, CrystalEyes shutter eyewear, Beverly Hills, CA, USA) to enhance depth perception, increase the sense of immersion and to facilitate normal upper extremity trajectories. The participant moves a virtual cursor through this space in order to touch ten targets presented randomly. Haptic assistance is provided if the subject is not able to reach a target within a predetermined time interval; 2) the goal of the “Placing Cups” task is to improve upper extremity range and smoothness of motion in the context of a functional reaching movement. The screen displays a three-dimensional room with a haptically rendered table and shelves (Figure 2C). The participants use their virtual hand (hemiparetic side) to lift the virtual cups and place them onto one of nine spots on one of three shelves. To accommodate patients with varying degrees of impairments, haptic effects like gravity and antigravity forces can be applied to the cups, global damping can be provided for dynamic stability and to facilitate smoother movement patterns, and the three dimensions of the workspace can be calibrated to adjust the range of motion required for successful completion of the task.

Hand and arm simulations

1) “Plasma Pong” trains upper arm and hand movement together. The pong paddle is moved with shoulder flexion and the target is engaged with finger extension, requiring the integration of shoulder flexion and finger extension. The trajectories of the target are non-predictable, thus necessitating constant conscious attention and feed-forward processing; 2) “Hummingbird Hunt”. This simulation depicts a hummingbird as it moves through an environment filled with trees, flowers and a river (Figure 3D). The “Hummingbird Hunt” provides practice in the integration of reach, hand-shaping and grasp using a pincer grip to catch and release the bird while it is perched on different objects located on different levels and sections of a 3D workspace. The flight path of the bird is programmed into three different levels (low, medium and high) allowing for progression of the arm and shoulder excursion required to successfully transport the arm to catch the bird; 3) “Hammer Task”. The “Hammer Task” trains a combination of three dimensional reaching and repetitive finger flexion and extension (Figure 3E). Targets are presented in a scalable 3D workspace. It exercises movement of the hand and arm together by having the subjects reach towards a wooden cylinder and then use their hand (via repeated finger extension) to hammer the cylinders into the floor. The haptic effects allow the subject to feel the collision between the hammer and target cylinders as they are pushed through the floor. Adjusting the length of the cylinders, the amount of antigravity assistance provided by the robot through the gimbal and the time required to successfully complete the series of cylinders, adaptively modifies the task requirements and game difficulty.

Figure 3.

Pre- to post-test percentage of improvement in the Jebsen Test of Hand Function and the Wolf Motor Function Test demonstrated by 4 subjects training hand and arm separately and 4 other subjects training their hand and arm together for eight, three hour sessions over a two week period (HAS: hand arm separately; HAT: hand arm togheter).

Subjects and procedure

All subjects signed institutionally approved consent forms. In one group we trained four subjects (mean age=51; years post stroke=3.5) approximately three h/day for 8 days on simulations that trained the arm and hand separately (HAS) (Piano 1, Reach/Touch, Space Pong, Placing Cups). In the second group, four other subjects (mean age=59; years post stroke=4.75) practiced for the same amount of time on simulations that trained the arm and hand together (HAT) (Piano 2, Plasma Pong, Hummingbird Hunt, Hammer).

Two types of outcome measures were used in this study. The primary dependent measures were the clinical tests; all subjects were tested pre- and post-training on, the Jebsen Test of Hand Function (JTHF) and the Wolf Motor Function Test. These tests were used to investigate whether improvement in hand and arm movement gained while training in the VE’s transferred to real-world tasks. The JTHF is a timed test developed to assess hand function and finger dexterity. There are seven functional subtests; writing, turning index cards, picking up small common objects, simulated feeding, stacking checkers, picking up large light objects, and picking up large heavy objects.²⁸ Our studies ²⁹ have confirmed that these tests appropriately distinguish between functional capabilities of the hemiparetic and non-hemiparetic hands, providing a valid discriminatory assessment tool. The Wolf Motor Function Test (WMFT) consists of 15 time-based functional upper extremity tasks such as, lifting and placing the hand on a box, folding a towel, lifting a soda can, picking up a pencil, and stacking checkers.³⁰

The secondary measures were the kinematic and force measurements derived from the VR system during training. These include time to task completion (duration), accuracy, velocity, smoothness of arm motion and force generated by the subject. The movement smoothness is evaluated through the normalized integrated third derivative of arm displacement (jerk).^{31, 32} Accuracy of key presses denotes the proportion of correct piano key presses.

Results

Figure 3 presents the percent change from pre- to post-training in the two primary clinical measures, the JTHF and the WMFT. The subjects who trained using their hand and arm together (HAT) showed greater improvement in their ability to perform the functional tasks than those subjects that practiced arm and hand tasks separately (HAS). The HAS group showed a 9% and a 14% improvement in the JTHF and in the WMFT whereas the HAT group showed a 29% and a 23% improvement, respectively. Table I indicates the percent change for each subject for the JTHF and the WMFT.

Table I.

Pre- to post-test percentage change in WMFT and JTHF for individual subjects.

	HAS				HAT
	S1	S2	S3	S4	S5	S6	S7	S8
JTHF	−87	8	5	38	37	22	29	23
WMFT	5	40	13	7	17	32	11	30

HAS: hand arm separately; HAT: hand arm togheter; JTHF: Jebsen Test and Hand Function; WMFT: Wolf Motor Function Test.

There were also notable changes in the secondary outcome measures; the kinematic data derived from the virtual reality simulations. Subjects in both groups showed similar improvements in the time to complete each game, 36–42% decrease depending on the specific simulation and in the smoothness of their hand trajectories, indicating better control.³³ However, the subjects in the HAT group showed a more pronounced decrease in the path length. This suggests a reduction in ineffective arm movements with more efficient limb segment interactions.

For training on the virtual piano simulations, subjects showed similar improvements in key press accuracy (percent change HAS=20%; HAT=17%). However, the subjects that trained using the arm and the hand together were able to complete the task much more quickly (percent change HAS=60%; HAT=151%).

Effects of sensory manipulations in VE on neural circuits

Methods

Subjects and procedure

Five right-handed,³⁴ subjects participated after signing institutionally approved informed consent forms. Four subjects (1 female, 3 males; mean age 26.8 years) were healthy with no history of neurological or orthopedic disease and participated in the functional MRI (N=3) and transcranial magnetic stimulation (N=1) experiments. The other subject was a 70-year-old right-handed woman who had a right hemispheric subcortical ischemic stroke 7 years ago. Although the patient’s function recovered considerably since then, her involved side remained significantly more impaired compared to her “non-affected” side (Jebsen test of hand function: right: 55, left: 151; strength dynamometry: right: 29 kg, left: 11 kg). The patient’s profile is detailed in Merians et al.²⁹

Each subject wore on their right hand a MRI-compatible 5DT Data Glove 16 MRI (Fifth Dimension Technologies, Irvine, CA, USA) with fiberoptic sensors that measured 14 joint excursions including the flexion-extension and abd/adduction of the metacarpophalangeal joints and flexion-extension of the proximal interphalangeal joints. The 5DT glove is metal-free and therefore safe to operate in a MRI environment.

Subjects performed an index-middle-ring-pinky finger sequence with their dominant hand (Figure 4A). Their finger motion animated the VE simulation, which consisted of real-time movement of either a virtual right hand (corresponding to their moving right hand) or a virtual left hand (corresponding to their stationary left hand). Additionally, to control non-specific effects related to the hemifield position of the object and motion, we added two control feedback conditions in which the virtual hands were replaced by a moving left or right non-anthropomorphic shape (ellipsoid) that was animated by a computer algorithm. The order of presentation of the conditions was randomized.

Figure 4.

A) The experimental setup. Subjects executed movements with their right hand which was measured by an fMRI-compatible data glove to animate motion of either the right or left virtual hand model; B) left panel. Activation, in a representative subject, is greater in the virtual mirrored relative to the virtual non-mirrored feedback condition. The activation in the ROI is rendered on an inflated cortical surface template using Caret software; B) right panel. The motor evoked potential plotted as a percent of the baseline MEP for the virtual mirror (black) and non-virtual mirrored (white) conditions. The two right bars show the MEPs when the virtual hands were shown and the two left bars show a control condition when the animated virtual hands were replaced with rotating ellipsoids; C) activation in a stroke patient for the same contrast shown for the control subject. The inset in the right panel shows the lesion location in the stroke patient.

Functional MRI (fMRI)

During the fMRI experiment, subjects lay supine wearing the data glove on their right hand. A set of fiberoptic cables (5 meters long) ran from the glove into the console room through an access port in the wall. In the console room, the fiber optic signals were digitized and plugged into the serial port of a personal computer that ran the VE simulation. The VE simulation was displayed to the subjects through a rear projector behind the magnet and the subjects viewed this through a rear-facing mirror placed above their eyes. Imaging was performed at NYU’s Center for Brain Imaging (3-T Siemens Allegra head-only scanner). We acquired a T1-weighted (1×1×1 mm³) 3D-MPRAGE pulse sequence structural image and T2*-weighted functional images (TR=2 500 ms, TE=30 ms, FOV=192 cm, flip angle=90°, bandwidth=4 112 Hz/px, echo-spacing=0.31 ms, 3×3×3 voxels, 46 slices). Preprocessing was done using SPM5 (Statistical Parametric Mapping. Software package available at: http://www.fil.ion.ucl.ac.uk/spm/). The first two volumes were discarded to account for field in homogeneities. Each subject’s functional volumes were realigned to the first volume, and the functional and structural images were then co-registered and spatially normalized to the Montreal Neurological Institute template. An event-related fMRI design was used and consisted of four functional imaging runs with 50 trials per run (total, 48 trials per condition). Condition-specific differences in the BOLD signal were analyzed with a general linear model approach for event-related fMRI using SPM5; activations were significant at a threshold magnitude of P<0.001 (FWE corrected) and extent of 10 voxels. Contrast images were analyzed on a subject-by-subject basis using a fixed effects model: (factors: left VR hand, right VR hand, left ellipsoid, right ellipsoid). Analysis was restricted to an a priori defined region of interest (ROI) in the right primary motor cortex. For this, we first created a ROI mask based on significant activation in a movement>rest contrast (ROI center for healthy subjects: 42 -12 64, radius: 20 mm; ROI center for stroke subject: 34 -18 64, radius 20 mm) using the “simpleROIbuilder” extension (SimpleROIBuilder. Software package available at: http://www-personal.umich.edu/~rcwelsh). We then applied the mask to the contrast of interest using the “Volumes” toolbox extension (SPM tools. Software packages available at: http://sourceforge.net/projects/spmtools). The control condition (the ellipsoid feedback) was subtracted from each virtual hand condition yielding our main contrast of interest: virtual left hand – left ellipsoid>virtual right hand – right ellipsoid. This contrast for the a priori defined ROI is shown for one healthy subject in Figure 4B.

Transcranial magnetic stimulation (TMS)

During the TMS experiment, the subject sat with both forearms resting on a table. Comfortable padding was placed under each forearm to elevate the wrist-hand off the table, allowing the subject to make finger movements while keeping the forearm stationary. An LCD monitor (Dell, 24 inch widescreen) was placed directly above and in-plane with the forearms, such that it was angled toward the subject’s head. The virtual reality models of hands were positioned in the LCD such that they overlay the subject’s actual hands (underneath the display) and were sized appropriately to the subject’s hands.

To measure corticospinal tract (CST) excitability, we applied single-pulsed TMS (Magstim Rapid2 stimulator with a 70 mm double coil) to the motor cortex ipsilateral to the moving right hand. A resting motor threshold was first determined as the minimum intensity sufficient to evoke a maximal contraction in the left first dorsal interosseus (FDI) muscle in 5 out of ten stimulations. The region in the right motor cortex at which the resting threshold was determined (i.e. the “hot-spot”) was marked on the subject’s high-resolution magnetic resonance image. The image was reconstructed into a 3D surface rendering using Brainsite (Rogue Research, Montreal, QC, Canada). The coil’s and subject’s head position were tracked throughout the session using optical cameras (Polaris camera, Northern Digital Inc., Waterloo, ON, Canada) and their positions were co-registered in real time with the subject’s MRI scan (using the Brainsite software). This allowed us to monitor and verify that the coil was aligned over the “hot-spot” throughout the entire session. TMS was applied at 110% of the motor threshold during each condition (see setup and procedure). Additionally, TMS was applied during a baseline rest condition (viewing a blank screen). Twelve trials were recorded for each condition. A 5-second inter-trial interval was used to prevent TMS spill-over effects from one trial to the next. To measure motor evoked potentials (MEPs), we recorded electromyographic activity in the FDI muscle with surface electrodes, amplified (band pass: 5 Hz-3 kHz), digitized (National Instruments, Austin, TX, USA 6036E card, sampling rate 1 200 Hz.), recorded (Labview Software, National Instruments) and saved for offline processing using Matlab software (MathWorks Inc., Natick, MA, USA). The raw EMG signal was rectified and smoothed with a 2^nd order Butterworth filter. The onset of each MEP epoch was defined 10 ms after the TMS artifact and the offset was defined as the time when the EMG signal returned to 5% of its peak for at least 100 ms. The maximal signal amplitude during this epoch was extracted for each trial and averaged across condition types. MEPs were normalized according to the formula [(x-y)/y]*100, where x is the mean MEP in a given condition and y is the MEP at baseline. Thus, the ordinate values in Figure 4B represent a percent change in each condition’s MEP relative to baseline.

Results

We hypothesized that mirrored feedback may enhance the excitability of the sensorimotor cortex. To test this, we manipulated feedback in a virtual environment in order to simulate mirror-reflected feedback of movement in the opposite hand. Figure 4B shows that virtual mirror feedback was associated with significant activation of the sensorimotor cortex contralateral to the mirrored hand (i.e. the cortex ipsilateral to the physically moving hand) (uncorrected P=0.016, number of voxels: 103). It is noteworthy that the subject’s physical hand movement was limited to the right hand only and remained constant in all of the conditions suggesting that our finding was attributed to sensory feedback manipulation rather than any differences in motor outpout. In a preliminary follow-up experiment, we tested whether the source of this increased activation was attributable to enhanced excitability of the corticospinal tract fibers by comparing single-pusle TMS-induced MEPs under the same conditions. We hypothesized that if the activation is due to CST facilitation, then TMS to motor cortex ipsilateral to the moving hand (i.e. contralateral to the mirror reflected hand) should lead to increased MEPs in the mirrored-feedback condition. Figure 4B shows that relative to baseline, MEPs increased by 71.4% and 63.2% when feedback was of the non-mirrored and mirrored virtual hands, respectively, but only by 15.3% and 19.1% in the respective control (ellipsoid) conditions. Additionally, the virtual mirrored hand condition led to an 8.2% stronger MEP (relative to the non-mirrored virtual hand condition). Conversely, no such facilitation was noted in the control (ellipsoid) conditions. Given our findings, and the emerging use of mirror therapy for rehabilitating hand-arm function in stroke patients, we investigated whether a similar facilitory effect in sensorimotor cortex would be observed in patients with neurological impairment. Figure 4C shows a stroke patient who performed our task with the same virtual mirror manipulation in an event-related fMRI design (see methods). Analysis of an a priori defined ROI centered on the sensorimotor cortex of the lesioned hemisphere (ROI center: 34 -18 64, radius 20 mm) revealed significant activation in the contrast “virtual left hand-left ellipsoid >virtual right hand-right ellipsoid”. In other words, moving the “unaffected” hand while viewing feedback of movement of the “virtual affected” hand (virtual mirror feedback) recruited the lesioned hemisphere (i.e. contralateral to the “virtual affected” hand).

Discussion

To demonstrate the feasibility of the system eight subjects were trained to use complex virtual environments integrated with haptic robotic devices. Each subject learned one of two different practice paradigms, hand and arm separate and hand and arm together. This initial study demonstrated that we have been able to develop a system that utilizes both virtual reality simulations and robotics to model rehabilitation that: 1) accommodates different levels of impairments; 2) provides haptic assistance to guide and modify the kinematics of the movement and 3) provides a practice environment that can train both the hand and the arm.

Our goal is to optimize training paradigms to enhance neuro-rehabilitation interventions. Upper extremity, and hand rehabilitation in particular, is a challenge in stroke rehabilitation. It remains controversial as to the most advantageous method of hand rehabilitation. Competition among neural representations ³⁵ would predict that training the arm and hand separately will promote better outcomes on motor recovery of the hand and improve transfer to real world movements. However, neural control mechanisms of arm transport and hand-object interaction are interdependent.³⁶ Recognizing the importance of training using functionally complex movements, one would predict that training the arm and hand together will promote more favorable outcomes. Our initial findings point to the possibility that training the arm and hand as a unit may provide a greater advantage for improving functional activities over training them separately. However we acknowledge that we only tested one of the possible training paradigms. We trained the arm and hand separately but on the same day; training them separately, for a longer period of time, arm first and hand after, would probably produce different outcomes. We plan to follow up on these mechanisms to determine the most advantageous training paradigm.

The use of virtual reality technology for rehabilitation has moved beyond simply providing a motivating gaming environment, suitable for intensive repetitive practice. Engineers in collaboration with clinical scientists are now developing “smart” simulations and robots that will work with the particular patient’s deficits to modify the kinematics of the movement. Newer robotic controllers are becoming adaptive to the performance of the patient’s motion.⁸ This offers the opportunity to integrate motor learning practices with mechanisms that help to adjust the kinematics and the forces of the movement during training. This integration allows patients with greater impairments to participate in a meaningful way and is very difficult to achieve during real-world rehabilitation. Our simulations use haptic assistance effects such as gravity and antigravity forces and damping for dynamic stability to modify the movement kinematics and encourage stable movement patterns.⁹ These technological interventions provide for physical assistance and graded progression of tasks.

Our data suggest that mirror virtual feedback can activate the sensorimotor cortex contralateral to the reflected hand (i.e. ipsilateral to the moving hand). This finding supports several recent TMS reports showing that MVF (using a real box-mirror setup) can lead to excitability of the motor cortex in amputee patients ³⁷ and in healthy subjects.^38–40 Interestingly, both our data and the data of these independent groups show that both mirrored and non-mirrored feedback seem to exert similar facilitation of the cortex ipsilateral to the moving hand. Although we noted an 8% increase in the mirrored condition, it must still be tested in larger sample sizes whether this is a significant and reliable effect. Additionally, the mechanism of this effect remains unknown. Visual input can provide a powerful signal for reorganization of sensorimotor circuits, Retrograde tracer studies show that rich intra-hemispheric cortico-cortical connections link the occipital, parietal, and frontal cortices ^41–47 and single unit data show that a substantial number of neurons in motor, premotor, and parietal areas are modulated by visual information.^48–52 Given the expanse of visual inputs to sensorimotor cortex, it is difficult to identify the sensory path that may be most susceptible to the effects of MVF. Although these issues remain to be resolved, it is indisputable that visual feedback can exert a strong modulatory influence over the motor system and can often override other afferent modalities, for example when a sensory conflict is introduced.⁵³ This may offer clinicians a powerful tool to facilitate neural recovery particularly in patients without the ability to produce overt movement (such as in the acute phase after stroke).

Conclusions

We have designed a series of gaming simulations interfaced with robotic devices. The purpose of these devices is to provide a training medium that can improve the movement kinematics by driving neural reorganization. Our VE therapies can optimize behavioral performance. Therefore, this study gives a preliminary evidence to support the potential of using specific VE-based sensory manipulations to selectively recruit targeted neural circuits.