Robotic Mirror Therapy System for Functional Recovery of Hemiplegic Arms

Abstract

Mirror therapy has been performed as effective occupational therapy in a clinical setting for functional recovery of a hemiplegic arm after stroke. It is conducted by eliciting an illusion through use of a mirror as if the hemiplegic arm is moving in real-time while moving the healthy arm. It can facilitate brain neuroplasticity through activation of the sensorimotor cortex. However, conventional mirror therapy has a critical limitation in that the hemiplegic arm is not actually moving. Thus, we developed a real-time 2-axis mirror robot system as a simple add-on module for conventional mirror therapy using a closed feedback mechanism, which enables real-time movement of the hemiplegic arm. We used 3 Attitude and Heading Reference System sensors, 2 brushless DC motors for elbow and wrist joints, and exoskeletal frames. In a feasibility study on 6 healthy subjects, robotic mirror therapy was safe and feasible. We further selected tasks useful for activities of daily living training through feedback from rehabilitation doctors. A chronic stroke patient showed improvement in the Fugl-Meyer assessment scale and elbow flexor spasticity after a 2-week application of the mirror robot system. Robotic mirror therapy may enhance proprioceptive input to the sensory cortex, which is considered to be important in neuroplasticity and functional recovery of hemiplegic arms. The mirror robot system presented herein can be easily developed and utilized effectively to advance occupational therapy.

Download video file ^{(33.3MB, mp4)}

Introduction

For patients with stroke, dysfunction of a hemiplegic arm has debilitating effect. The ability to perform bimanual activities is essential for daily life, but functional deficit of a hemiplegic arm often remains even a few years after stroke onset. Among various training programs in the hospital, an exercise to increase the range of motion or passive repetition of simple tasks have little effect on functional recovery of a hemiplegic arm. For this reason, training of meaningful tasks related to activities of daily living (ADLs) has been applied to occupational therapy in hospitals.

The effects of mirror therapy were proven by previous studies in neurorehabilitation^1-4. Mirror therapy is conducted by eliciting an illusion through use of a mirror as if the hemiplegic arm is moving in real-time while moving the healthy arm. It can facilitate brain neuroplasticity by activation of the sensorimotor cortex¹. Thus, motor power and function of the hemiplegic arm can be improved. However, conventional mirror therapy has a critical limitation in that the hemiplegic arm is not actually moving.

Therefore, we developed a real-time 2-axis mirror robot system as a simple add-on module to conventional mirror therapy, using closed feedback mechanism. This may convey proprioceptive input to the sensory cortex, which is considered important in neuroplasticity and functional recovery of a hemiplegic arm (Figures 1 and 2)^5-7.

Protocol

 All of the procedures were reviewed and approved by the Institutional Review Board of Seoul National University Hospital.

1. Mirror Therapy Tasks

1. Examples of 2-dimensional mirror therapy tasks (Figure 3)

   1. Freely move the healthy arm while looking in the mirror about 5 min for warm-up exercise. NOTE: One may utilize a metronome so that the patient can exercise the motion of the healthy arm in a rhythmic manner.
   2. On healthy side, dribble and place a small ball into the chosen hole similar to billiards for about 5 min ("Ball in holes" task). Dribble and place a small ball into a goal similar to soccer for about 5 min ("Soccer game" task).
   3. Using numbered stickers placed on a table, move the handle on healthy side in numerical order and return in reverse direction ("Dots tracing" task). Repeat for about 5 min.
   4. Using any object in daily life such as a cup, using handle on healthy side, push it to a chosen place ("Moving a cup" task). Repeat for about 5 min.

2. Components of the Mirror Robot System

1. AHRS sensor settings

   1. Obtain 3 commercially available AHRS sensors. NOTE: AHRS sensors consist of a magneto sensor, accelerometer, and gyro sensors (total 9-axis).
   2. Connect the AHRS sensor to a PC with a USB connector.
   3. Use hyperterminal or other communication software to configure general sensor settings.
   4. For each AHRS sensor, set to RS232 communication and select COM port. Then, set baud rate to 115,200 bits per sec, data bits to 8, parity to none, stop bits to 1, and flow control to none.

      1. To check the COM port, click home button on bottom left corner. Right click on computer. Then click Properties. Click Device Manager. Expand Port (COM & LPT) tab by clicking it.
   5. Once the communication has been established, set channel to 100 and assign IDs for each sensor. NOTE: Some sensors may need calibration of the accelerometer, gyroscope, and magnetometer before use.
   6. Set output format as quaternions and set sensors to display battery reserve. NOTE: Quaternions are used to speed up computing as well as to eliminate gimbal lock singularities.
2. Brushless DC motor settings

   1. Prepare 2 high-performance brushless DC motors and controllers.
   2. For each controller, connect the power cable to a power supply. Also, connect the motor cable, Hall sensor cable, and encoder cable to the motor.
   3. Connect CAN-CAN cable to another controller. NOTE: CANopen is used for communication between devices.
   4. Set node ID for each controller to distinguish between devices.
   5. Connect the USB cable to the PC for general configuration.
   6. Switch on the power supply to power up the controllers and motors.
   7. Use the motor manufacturer-provided system configuration software to configure and tune the motor, Hall sensor, and encoder. NOTE: Angle limits and home position must be configured for safe operation.
3. Assembly of frame and motors NOTE: Every custom-made part is named in quotation mark. Please refer to the Table of Materials & Equipment and from Figure 4 to Figure 13.

   1. For the elbow joint motor, put one of the coupling bodies with keyway on the motor shaft and secure it using an M5 hex socket set screw (Figure 4).
   2. Secure "Elbow coupling hollow cylinder cover" to the elbow motor using 4x M5 socket head screws (10 mm) and place the buffer part of the couplings (middle slider part) on the top of the coupling body that was attached in step 2.3.1 (Figure 4).
   3. Plug the ball bearing into "Elbow rooftop frame" and secure it with 4x M4 socket head screws (8 mm) (Figure 5).
   4. Plug "Elbow motor force dispersion shaft" into "Lower elbow support" and secure it with 4x M3 socket head screws (6 mm). Then, place "Upper elbow support" on top of the "Lower elbow support" and secure it using 8x M3 socket head screws (12 mm) (Figure 6).
   5. Place the assembly in step 2.3.4 on top, the assembly in step 2.3.3 in the middle, and the last part of the coupling body at the bottom. Join all together and secure the coupling body with M5 hex socket set screws (10 mm) (Figure 7).
   6. Secure assembly in step 2.3.5 and assembly in step 2.3.2 using 4x M5 socket head screws (15 mm) (Figure 7). Rotate assembly in step 2.3.2 to secure all 4 points.
   7. Secure "Lower wrist coupling hollow cylinder cover" with the wrist motor using 4x M4 socket head screws (10 mm). Then, place one of the coupling bodies with keyway on the motor shaft and secure it using M4 hex socket set screws; then, place the buffer part of the couplings on top of the coupling body (Figure 8).
   8. Attach "Friction reduction ring" on top of "Wrist rooftop frame" with double-sided tape or any type of adhesive (Figure 9).
   9. Plug "Wrist motor force dispersion shaft" into "Handle" and secure it using 4x M2.5 socket head screws (4 mm) (Figure 10).
   10. Place assembly in step 2.3.9 on top, assembly in step 2.3.8 in the middle, and the last part of the coupling body at the bottom. Join all together and secure the coupling body with M4 hex socket set screws (10 mm) (Figure 10).
   11. Secure "Wrist motor2roof2" with the assembly in step 2.3.10 using 4x M3 socket head screws (Figure 11).
   12. Secure assembly in step 2.3.11 and assembly in step 2.3.7 using 4x M3 socket head screws (15 mm) (Figure 11).
   13. Secure 2 "Joint movement limiter" and 2 shaft collars using 4x M4 socket head screws (15 mm) (Figure 12A).
   14. Use shaft collars to secure shafts and "Wrist rooftop frame" using 8x M3 socket head screws (8 mm) (Figure 12B).
   15. Slide shaft collars in assembly 2.3.13 into the shafts in assembly 2.3.14 and secure additional shaft collars with "Lower elbow support" using 4x M4 socket head screws (15 mm). Then, join the two parts and secure with lever (Figure 13A).
   16. Secure "Support wall" to the assembly in step 2.3.15 using 6x M4 socket head screws (15 mm) (Figure 13B). Secure the table stand and assembly in step 2.3.16 using 6x M6 socket head screws (15 mm) (Figure 13C).

3. Design of Mirror Robot System

1. Mathematical model for automatic control

   1. Set dynamic model for automatic control of upper limb motion (Figure 14). NOTE: A dynamic model of human upper limb motion can be expressed using kinematics of joints and links. Therefore, using an equation for the robot manipulator, the modeling can be obtained as shown below: NOTE: (: Joint position vector, : Joint velocity vector, : Joint acceleration vector, H: Inertia matrix, F: Coriolis and centrifugal force matrix, G:Vector of gravitational forces, E: Torque matrix due to interaction with the environment, : Vector of generalized forces applied to the joints) The hemiplegic and healthy arm show different aspects of motion. That is, the hemiplegic arm cannot move in time due to paralyzed muscles or cannot provide enough torque needed for motion. Therefore, the system is designed such that the rehabilitation training can be done via normal motion through the hemiplegic arm; in other words, the rehabilitation robot is attached to the patient's hemiplegic arm in order to deliver motions from the healthy arm and can be simply formulated as follows: Motion of rehabilitation robot) = (Healthy arm motion) - (hemiplegic arm motion).
   2. With a rehabilitation robot, attach the patient's paralyzed arm to the manipulator, and observe additional torque and time delay due to the paralyzed arm causing errors in the overall system. Detect this via a manipulator on the hemiplegic side.
   3. Measure errors (s(t): tracking error) as a mathematical equation: NOTE: (s: Tracking error, : Positive definite design parameter matrix, : Error between desired and actual position, : Error between desired and actual velocity) The above tracking error can be combined with a dynamic model of human upper limb motion and can be expressed as: NOTE: (K_D : Derivative gain value with feedback compensation that changes over time, : Inertia error matrix, : Coriolis and centrifugal force error matrix)
   4. To control each joint of the rehabilitation robot, use Lagrangian dynamics⁸. A dynamic equation of motion for each joint is: NOTE: (D: Coefficient matrix, : Actuator inertia matrix) Coefficient D in the equation above affects the torque between joints with inertial coupling effect between joints⁸. The automatic control model using this mathematical model can be illustrated by the block scheme in Figure 14.
2. Software protocol (Figure 15)

   1. When the program starts, establish communication with motors and sensors, and initialize the values. Once the motors and sensors are at the initial position (see 4.1.3), move on to the main loop. NOTE: For sampling frequency of the main loop, we recommend 50 - 200 samples/sec. For the maximum delay, we recommend 2 sec at most. In addition, for torque limits, we recommend to regulate the motor current value with software so that the elbow motor can exert 25 - 40 Nm and wrist motor can exert 10 - 20 Nm.
   2. As it is not interrupted by a stop button, continuously read the Attitude and Heading Reference System (AHRS) sensors' current position values to transmit the values to the motors. NOTE: The data output is in quaternions, and should be properly transformed into the desired angle for robot motion. Choose one of the sensor coordinate frames as a reference, and reset the other sensor coordinate frames. With calculated frames as a reference, use inverse kinematics to obtain the final output yaw angles.
   3. As it is not interrupted by a stop button, continuously check the motors' positions and update the values to accomplish movement to the desired position provided by the AHRS sensors. NOTE: The motor position is provided by motor's encoder that can be checked inside the software with the motor company's provided software library command.
   4. Meanwhile, record all angles and angular velocities from the AHRS sensors.
   5. Once the tasks are completed and the user presses the stop button, exit the loop and finalize the robot by moving it to the initial position.
3. Graphical user interface (GUI) (Figure 16)

   1. Add "error in" and "error out" functions to detect and debug errors during execution.
   2. Add patient-side button to choose the robot operation side (patient's paretic side).
   3. Construct a patient information box to identify the patients.
   4. Add motor status indicators.
   5. Add angle limitation controls for safety.
   6. Configure the maximum velocity, acceleration, and deceleration for each motor to prevent muscle and tendon injuries due to a stiff upper limb. NOTE: The system reflects the acceleration and deceleration of the hemiplegic arm.
   7. Add indicators to retrieve motor position and velocity, and input current information.
   8. Build a VISA resource name control to establish communication between AHRS sensors and the system.
   9. Add a calibration function to eliminate accumulated sensor drift errors.
   10. Arrange the indicator for the sensors in order to retrieve sensor information. NOTE: The sensor information includes the joint angles (angle between two consecutive sensors) and battery reserve.
4. Overcoming arm spasticity during mirror robot operation

   1. Select motors that can exert enough torque to overcome spasticity for each joint. NOTE: Wrist motor should have torque output higher than 10 Nm, and elbow motor higher than 25 Nm.
   2. In order to firmly transfer the robot motion to the patient's arm, use straps that are made of semi-elastic material to fix the forearm in the robot exoskeleton. NOTE: Semi-elastic straps, such as stretch fabrics straps or polyester/nylon elastic braided straps, are recommended. If the straps are too elastic, it will not hold the arm in position. If the straps are not elastic at all, muscle or tendon injuries may occur in the case of a high degree of elbow spasticity.
   3. In order to isolate elbow and wrist motion, use 2 solid frames combined with a shaft collar to fix the wrist by squeezing it in the frames. NOTE: Shaft collars are used to prevent muscle and tendon injuries if the stiffness in the wrist is excessive.
   4. Use straps around the handle to fix the hand to the robot.

4. Clinical Application of the Mirror Robot System

1. Conducting robotic mirror therapy

   1. Adjust height and width of the task table in accordance with the patient's condition.
   2. Set up a mirror in the midline between both arms, and set it on a table or platform.
   3. Place AHRS sensors on handle, wrist frame, and edge of platform on healthy side aligning in parallel with the robot's orientation. NOTE: The sensor's internal yaw axis should be pointing up.
   4. Execute the therapy software in a computer.
   5. Choose the hemiplegic side by clicking the Patient-Side switch button.
   6. Set the maximum joint angle limits in accordance with patient's joint condition. For safe operation, use elbow flexion limit less than 50º, elbow extension limit more than -70º, wrist flexion limit less than 80º, and wrist extension limit more than -60º. NOTE: Plus and minus signs are automatically corrected and limits are also corrected if out of bounds in software level.
   7. Set maximum velocity, acceleration, and deceleration. For these values, use velocity value between 0 and 22.5 rpm for elbow motor and use velocity value between 0 and 33 rpm for wrist motor. NOTE: For conventional mirror therapy, set all values to zero to immobilize the robot.
   8. Fill out patient information.
   9. Turn on all AHRS Sensors before running the program.
   10. Run the program by clicking the arrow button in the upper left corner of the program.
   11. Once the "save as" prompt pops up, write proper file names for result data on the string box and press ok.
   12. While the robot and the healthy arm are at initial position (both hands away from the body and parallel to each other), press the calibration button to initialize sensor values to zero for the initial position. NOTE: See steps 1.1.1 - 1.1.4 for the hands used in this task.
   13. Press STOP button when all the tasks are completed. NOTE: For the robotic mirror therapy, a biomedical engineer should act as the main coordinator, and the occupational therapist should assist the patient.
2. Clinical a study on healthy subjects

   1. Conduct a clinical study on healthy subjects to confirm safety and feasibility⁸. Give the instruction ("Do not move your hemiplegic arm on your own.") to the subjects for completely passive movement of hemiplegic arm.
   2. Place both forearms on the frames, and hands on the handles. Then, fix the forearms with straps.
3. Evaluation of therapeutic effects

   1. Before the therapy, conduct functional evaluations such as Fugl-Meyer assessment scale⁹, modified Ashworth scale¹⁰, modified Barthel index¹¹, Jebsen hand function test, hand power measurement, hemispatial neglect test, and motor evoked potential test for the patients.
   2. Conduct clinical trial for stroke patients with 2-dimensional mirror robot for 30 - 60 min per day. Give the instruction ("Do not move your hemiplegic arm on your own.") to the patients.
   3. After the patients accomplish last session, conduct follow-up functional evaluations.

Representative Results

Six healthy subjects conducted a 'pen marking task' (touching the two small boards alternately with a pen attached on the healthy hand as shown in Figure 17) 10 times which took on average 106 sec per subject. No adverse event was observed, and robotic mirror therapy was proven to be feasible.

In addition, a clinical study on rehabilitation doctors was conducted. We requested expert opinions to determine appropriate tasks for effective robotic mirror occupational therapy. With feedback from 6 rehabilitation doctors, the degree of illusion elicited by the mirror robot was highest for "ball in holes" and "moving a cup" tasks (7.2 out of 10 on a numerical rating scale [NRS] for each), followed by "soccer game" (7.0/10) and "dots tracing" tasks (6.5/10). Regarding the synchronicity of movement between both arms during robotic mirror therapy, "moving a cup" task had an NRS score of 7.0/10, followed by "soccer game" and "dots tracing" (6.8/10 each), and "ball in holes" (6.2/10) (Figure 3). Among these 4 tasks, rehabilitation doctors recommended "soccer game" as a useful task for ADL training in patients with stroke.

We conducted a clinical trial for stroke patients with mirror robot for 30 min per day for 2 weeks (10 sessions). Subjects need to meet the following inclusion criteria: 1) over 18 years old; 2) supratentorial stroke diagnosed between 4 months and 6 years ago; and 3) upper-limb hemiplegia with Medical Research Council (MRC) grade 2 or less. Main exclusion criteria are as follows: 1) modified Ashworth scale of grade 3 or more (severe spasticity); 2) mini-mental state examination score less than 12; and 3) global or sensory aphasia.

A 60-year-old male stroke patient with right basal ganglia hemorrhage and left hemiplegia 19 months prior received robotic mirror therapy for 2 weeks. Robotic mirror therapy consisted of 4 tasks ("ball in holes", "moving a cup", "soccer game", and "dots tracing"). After the patient accomplishes the 10^th visit, he conducted follow-up functional evaluations. The Fugl-Meyer assessment scale of the hemiplegic arm improved from 12 to 17 out of 66, and modified Ashworth scale of elbow flexors (for spasticity) was reduced from grade 2 to 1+. Left lateral pinch power was increased from 0 to 3 lb. Other parameters revealed no difference before and after robotic mirror therapy (Figure 18 and Table 1).

Figure 1.Conceptual Flow for the Robotic Mirror Therapy to Facilitate Proprioceptive Input. The experiment is designed in accordance with the conceptual flow for the robotic mirror therapy.

Figure 2.A Diagram of the Mirror Robot System. Movements of the healthy arm are projected to the exoskeleton attached to the hemiplegic arm by a software algorithm through input from 3 AHRS sensors. Please click here to view a larger version of this figure.

Figure 3.Various Tasks using the Mirror Robot System. The users can be trained by 2-dimensional tasks; ball in holes, soccer game, dots tracing, and moving a cup. Please click here to view a larger version of this figure.

Figure 4.Elbow Motor Assembly. Assembly steps for elbow joint motor, couplings, and elbow coupling hollow cylinder cover.

Figure 5.Bearing & Elbow Rooftop Frame Assembly. Assembly between bearing and the elbow rooftop frame assembly.

Figure 6.Elbow Support Assembly. Assembly steps for elbow motor force dispersion shaft, upper elbow support, and lower elbow support.

Figure 7.Elbow Support & Elbow Motor Assembly. Assembly steps for the elbow support and the elbow motor.

Figure 8.Wrist Motor Assembly. Assembly steps for wrist joint motor, couplings, and lower wrist coupling hollow cylinder cover.

Figure 9.Friction Reduction Ring Attachment. Attachment of the friction reduction ring to the wrist rooftop frame.

Figure 10.Handle Assembly. Assembly steps for the 3D printed handle, coupling, and the wrist motor force dispersion shaft.

Figure 11.Handle & Wrist Motor Assembly. Assembly steps for the wrist motor and the handle.

Figure 12.Joint Movement Limiter Assembly. Assembly steps for the (A) joint movement limiter, (B) length adjustment shaft, and the assembled handle.

Figure 13.Final Assembly. Assembly steps for the (A) assembled elbow motor part with the assembled wrist motor part using shaft collars and shaft, (B) assembled robot with the support walls, and (C) assembled robot with the task table. Please click here to view a larger version of this figure.

Figure 14.Block Scheme of the Automatic Control Mathematical Model. The exoskeleton robot utilizes closed feedback mechanism for real-time control.

Figure 15.Overall Software Program. The software program uses a closed feedback mechanism to drive the robot system. Please click here to view a larger version of this figure.

Figure 16.GUI of the Program. User can control and configure the program for therapy via GUI. Please click here to view a larger version of this figure.

Figure 17.A Pen Marking Task in 6 Healthy Subjects using Prototype of Mirror Robot System. Conducting a pen marking task 10 times consecutively took on average 106 sec per subject.

Figure 18.Functional Evaluation of a 60-year-old Male Patient with Chronic Right Basal Ganglia Hemorrhage. Main subsets of data that showed improvement after 10 sessions of robotic mirror therapy. Please click here to view a larger version of this figure.

	Before	After 10 sessions
Mini-mental state examination	29	-
Fugl-Meyer assessment scale (Upper extremity)	12	17
Shoulder/elbow	11	15
Wrist	0	1
Hand	1	1
Modified Ashworth scale
Elbow flexor	2	1+
Wrist flexor	0	0
Modified Barthel index (Upper extremity)	25	25
Jebsen hand function test	Uncheckable	Uncheckable
Left hand power (lb)
Grip	8	8
Lateral pinch	0	3
Palmar pinch	0	0
Hemineglect test
Line bisection test	6/6 each	6/6 each
Albert test	12/12 each	12/12 each
Motor evoked potential	No response	No response

Table 1. Functional Evaluation of a 60-year-old Male Patient with Chronic right Basal Ganglia Hemorrhage.

Discussion

The primary purpose of this study was to develop a real-time mirror robot system for functional recovery of a hemiplegic arm using an automatic control algorithm. The effect of robot-assisted therapy on long-term recovery of upper-limb impairment after stroke was proven beneficial in previous studies¹², and various kinds of arm robots have been introduced^13-20. However, previous studies of upper extremity robots that realized bilateral arm movement applied mechanical connections without using a mirror, which is different from the concept of mirror therapy^14-15. Thus, our study can be an extension of their work by using an actual mirror to facilitate proprioceptive input.

To upgrade the previous system, we enabled the hemiplegic arm to move in real-time by applying AHRS sensors on the healthy arm and attaching motors to the hemiplegic elbow and wrist. Proprioceptive input from the hemiplegic arm to the sensory cortex of brain can be enhanced through the mirror robot system. Facilitation of proprioception needs to be confirmed by functional brain MRI in a future study.

It is critical for the system to have minimum synchronization delay since the mirror effect will be maximized when the delay is minimized. To achieve this, we retrieved data from sensors with minimum necessary byte count while reading them in parallel within a loop inside the software architecture. As a result, the synchronization delay between the healthy arm and the robot is only about 0.04 - 0.40 sec.

There are several limitations in this study. First, we could not include fine finger movements such as grip or pinch, and 3-dimensional tasks of conventional mirror therapy. Second, we did not fix the elbow joint of the healthy arm to preserve physiologic movement as much as possible. However, restriction of the range of elbow motion would be helpful to enhance synchronicity with the opposite elbow which is moved by the motor. Modifying the system by installing additional structure that secures the healthy side elbow will improve the synchronicity and, therefore, will increase the effect of the therapy. Third, patients who had severe spasticity or stiffness could not be included because of insufficient motor power, although the joint moved slowly. The system could be modified by replacing the motor with higher torque output to overcome moderate stiffness. However, even with strong motor, treatment to patients with severe levels of spasticity or stiffness should be avoided to prevent tendon or bone injuries due to excessive force application to the joints.

We believe, however, that the mirror robot system presented herein can be easily developed and utilized effectively to advance occupational therapy.

Disclosures

The authors have nothing to disclose.