A low cost kinect-based virtual rehabilitation system for inpatient rehabilitation of the upper limb in patients with subacute stroke

Supplemental Digital Content is available in the text

Abstract

Background:

We designed this study to prove the efficacy of the low-cost Kinect-based virtual rehabilitation (VR) system for upper limb recovery among patients with subacute stroke.

Methods:

A double-blind, randomized, sham-controlled trial was performed. A total of 23 subjects with subacute stroke (<3 months) were allocated to sham (n = 11) and real VR group (n = 12). Both groups participated in a daily 30-minute occupational therapy for upper limb recovery for 10 consecutive weekdays. Subjects received an additional daily 30-minute Kinect-based or sham VR. Assessment was performed before the VR, immediately and 1 month after the last session of VR. Fugl-Meyer Assessment (FMA) (primary outcome) and other secondary functional outcomes were measured. Accelerometers were used to measure hemiparetic upper limb movements during the therapy.

Results:

FMA immediately after last VR session was not different between the sham (46.8 ± 16.0) and the real VR group (49.4 ± 14.2) (P = .937 in intention to treat analysis). Significant differences of total activity counts (TAC) were found in hemiparetic upper limb during the therapy between groups (F_2,26 = 4.43; P = .22). Real VR group (107,926 ± 68,874) showed significantly more TACs compared with the sham VR group (46,686 ± 25,814) but there was no statistical significance between real VR and control (64,575 ± 27,533).

Conclusion:

Low-cost Kinect-based upper limb rehabilitation system was not more efficacious compared with sham VR. However, the compliance in VR was good and VR system induced more arm motion than control and similar activity compared with the conventional therapy, which suggests its utility as an adjuvant additional therapy during inpatient stroke rehabilitation.

1. Introduction

Upper limb weakness is the most common impairment after stroke^[1] and is associated with the need for assistance in activities of daily living and worsened quality of life.^[2] Although motor recovery continues to improve up to 1 year after stroke, active recovery usually occurs around 3 to 6 months post-stroke.^[3,4] The time window for the recovery for motor control and through neural plasticity is short and active rehabilitation strategies in the subacute stage is considered to be important.^[5–7] Based on motor learning theory, task-oriented, intensive, and repetitive training during subacute stage after stroke is the key factor for promoting neural plasticity to induce motor recovery.^[8] Increased therapy is related with greater motor functional improvement after stroke.^[9] However, the amount of occupational therapy is usually less than the suggested guideline,^[10] and active participation during conventional therapy is also less than the expected.^[11,12]

Virtual rehabilitation (VR) is one of the candidate modalities for motor rehabilitation to induce active participation and more repetitions by motivating patients.^[13] Although a recent multicenter randomized controlled trial using a commercial gaming system failed to show the superiority to conventional occupational therapy,^[14] the VR system designed to deliver task-specific training for stroke patients can have beneficial effects on motor recovery.^[15] However, few studies with relatively few patients have compared the performance of VR systems designed for stroke motor rehabilitation. Furthermore, most of the studies did not focus on the subacute stage of stroke.^[16]

We designed this study to prove the efficacy of our developed low-cost Kinect-based VR system for upper limb recovery among patients with subacute stroke, compared with sham VR. In addition, because we postulated that one of possible advantages of VR system may be a reduction on the intervention time by the therapists while maintaining a similar amount of arm movements during occupational therapy, we measured the amount of hemiparetic upper limb movements during real and sham VR by using an accelerometer, and the amount of time intervened by the therapists during real VR, to investigate whether our VR system has the potential to be used as a group therapy tool in the inpatient stroke rehabilitation setting.

2. Methods and materials

2.1. Subjects

Subjects were recruited from May 2014 to November 2016. Subjects were eligible for inclusion in the study if they had acute (<3 months), first-ever ischemic or hemorrhagic stroke confirmed by magnetic resonance imaging or computed tomography, with unilateral upper extremity weakness, and who could perform the reaching activity with their hemiparetic arm. Subjects were excluded from the study if they were younger than 20 years of age or older than 80 years of age; uncontrolled medical condition (e.g., active infection, pulmonary embolism, unstable angina), were unable to follow verbal commands due to cognitive impairment or aphasia, were unable to participate Kinect-based VR due to any reason (e.g., visual impairment, hemispatial neglect, apraxia), or could not provide written informed consent. All subjects received detailed information about the study and provided their written consent. This research protocol was approved by the Seoul National University Bundang Hospital institutional review board and was conducted in accordance with the regulatory standards of Good Clinical Practice and the Declaration of Helsinki (World Medical Association Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects, 2000). The trial was registered with the National Institute of Health ClinicalTrials.gov protocol registration system (registration number: NCT02066116). Planned sample size was 40 but we decided to interrupt the study because for another clinical trial to see the effect of intervention for upper limb recovery in stroke patients was being conducted simultaneously, which made further enrollment in our trial difficult, and because of the pilot nature of our initial plan, we decided that the number of enrollments was sufficient at the time of termination.

2.2. Experimental design

A double-blind, randomized, sham-controlled trial was performed. Subjects were randomized 1:1 to receive either real or sham VR. Randomization sequence was generated by a computer and concealed using opaque envelopes. This procedure was done by the investigator (NJP) who was not involved in the selection, intervention, and assessment of patients. Both subjects and assessors were blinded to group allocation. Both the real and sham VR groups participated in a daily 30-minute occupational therapy session targeting the hemiparetic upper limb recovery based on the adaptive task practice (shaping)^[17,18] for 10 consecutive weekdays (5 days per week). The real VR group received an additional 30 minutes daily VR using our developed Kinect-based VR system and sham VR group received additional 30 minutes daily of sham therapy using RehaCom (Hasomed Inc., Magdeburg, Germany), which is used mainly for cognitive training but requires some upper extremity motion during the therapy.^[19] All participants received conventional rehabilitation service according to their other impairments such as gait training, swallowing training, and speech therapy during admission. Assessment was performed before the VR, immediately, and 1 month after the last session of VR by 1 occupational therapist blinded to the group allocation. Subjects were not informed of the group assignment and were not allowed to discuss the VR they received with other patients or occupational therapists.

2.3. Kinect-based VR system

We developed three types of programs: “Push Museum,” “Apple Run,” and “Fruit Market.” These programs were made using the Unity three-dimensional (3D) game engine (Unity Technology Inc., San Francisco, CA). Push Museum uses the Microsoft Kinect Sensor (Microsoft Corporation, Redmond, WA) and its software development kit (SDK). Apple Run and Fruit Market use Apple Primesense Carmine 1.09 sensor (Apple Inc., Cupertino, CA) and Nimble SDK (Occulus VR, LLC, Irvine, CA) for the hand tracking. Our system could induce arm motions important during rehabilitation (reaching, wrist extension, hand grasping, and releasing) according to the subject's functional status (Fig. 1).

Figure 1.

Introduction of games and its training site.

Camera 1 (Fig. 2) is positioned 0.7 to 1 m from the desk to track the fine motor information tracked by the Nimble SDK, which is transmitted to the Apple Run and Fruit Market via the internal network. In Apple Run, the wrist angle is measured and used for the Avatar control. The wrist angle uses the roll, pitch, and yaw angles between the Wrist to palm center vector and the fixed coordinate system. The pitch angle is used for the vertical action game and the yaw angle is used for the horizontal action game. In Fruit Market, a linear discriminant classifier^[20] is applied to distinguish between grasping and non-grasping. Camera 2 (Fig. 2) is fixed to the desk and the user must be located at least 1 m from the camera. Of the full body joint information obtained from the Microsoft Kinect SDK, upper data from the shoulder center to the hand are used for Push Museum. The in-game position of the push controller is the converted position of the hand point in the shoulder center coordinate system, calculated as: 

Figure 2.

Hardware installation (side view). Camera 1 is used for “Apple run” and “Fruit Market.” Camera 2 is used for “Push Museum.”

where P_h is the hand position, P_c is the controller's position, and T_s.c is the shoulder center's coordinate system. T_s.c is 4 × 4 affine transformation matrix, which consist of unit vector of 3 axis and position of shoulder center. P_h and P_c are 4 × 1 vector. The detailed information for the gaming system is further explained in the supplementary file.

2.4. Intervention

Subjects sat comfortably in a chair during the VR session. Before starting the VR session, a 3-axis accelerometer (ActiGraph GT3X; Actigraph Corporation, Pensacola, FL) was attached to the wrist in the hemiparetic side using a strap. VR was implemented by the designated occupational therapists. For the real VR session, therapists selected the appropriate gaming contents and difficulty level according to the subject's upper limb function and set up to start the selected VR. Therapists tried to minimize the therapeutic handling technique during VR. The allowed interventions during VR was to explain the game rules again, adjust the Kinect angle and the distance from Kinect to subjects, fix the unpredictable system errors, change the gaming contents and difficulty level during the session and monitor the safety. For the sham VR session, the similar strategy as for real VR was applied and subjects were instructed to use the hemeparetic upper limb. Therefore, most of the activities in the hemiparetic upper limb were reaching to the button and pushing the selected button during the cognitive task. If pushing the button was difficult, therapists were allowed to help subjects.

2.5. Functional outcome measurements

The subjects were functionally assessed at baseline and immediately and 1-month after the last session of VR. The primary outcome was Fugl-Meyer assessment (FMA) immediately after the last session of VR. The FMA for upper limb function comprised 33 items, with score ranging from 0 to 66, with higher scores indicating better function.^[21] FMA assessed 1-month after VR was used as one of the secondary outcomes.

Other functional measurements as secondary outcomes were Brunnstrom stage for arm and hand, Box and Block test (BBT) in the hemiparetic arm, and Korean version of modified Barthel index (K-MBI). Brunnstrom stages comprised of 6 grades ranging from 1 (flaccid) to 6 (near normal), representing the motor recovery after stroke.^[22] The BBT evaluates the patients’ abilities to grasp and carry items. BBT was performed according to the previously published instructions. Subjects were instructed to move as many blocks as possible from one compartment to the opposite compartment for 60 seconds.^[23] The total number of blocks moved was scored. K-MBI evaluates the basic activities of daily living with the score from 0 (totally dependent) to 100 (independent).^[24]

2.6. Accelerometer data

Accelerometer data using ActiGraph from the hemiparetic upper limb were obtained during the VR session to measure the degree of activity during the therapy session. The acceleration was recorded along 3 axes with 30 Hz frequency. The data were downloaded using ActiLife 6 software (Actigraph Corporation, Pensacola, FL) with band-pass filtered data between frequencies of 0.25 and 2.5 Hz, using a proprietary process to remove acceleration due to gravity, down-sample data to 1 Hz samples, and convert acceleration into activity counts. Total activity counts (TAC) during each VR session, which was a composite vector magnitude from 3 axes, were calculated. TAC calculated in the last VR session was used for the statistical analysis. Upper limb activity measurements using a wrist-worn accelerometer in stroke patients has demonstrated good validity and reliability.^[25,26]

For the control data of TAC for comparison, we used the data from longitudinal observational study (from April 2015 to August 2015) to monitor the activity using a accelerometer in stroke patients during inpatient rehabilitation, which was also approved by the Seoul National University Bundang Hospital Institutional Review Board. Data from a total of 12 patients during 30 minutes occupational therapy for the upper limb recovery were used and TACs in the hemiparetic upper limb were calculated.

2.7. Direct intervention time during VR by therapists

Occupational therapists recorded their time to guide subjects during VR with stop watch. The average intervention time for 10 sessions of VR for each subject was calculated and used for the analysis.

2.8. Subject's guess for the group allocation

Subjects were asked to select their allocation groups at the end of last session of VR: sham VR versus real VR for upper limb rehabilitation.

2.9. Statistical analyses

Continuous variables are presented as means ± SD. Categorical variables are presented as frequencies (percentages). To compare the baseline characteristics between real and sham VR groups, Student t test was used for continuous variables and chi-square test was used for categorical variables. An uncorrected 2-tailed P < .05 was considered statistically significant. To compare the baseline characteristics and TAC between 3 groups (sham VR, real VR, and control), analysis of variance (ANOVA) was used. Sheffe test (with a significant P < .05) was used as a post-hoc test when ANOVA showed significant differences (P < .05).

Both per-protocol and intention-to-treat analyses were used in analyzing primary and secondary outcomes. For the per-protocol analysis, patients who did not complete the entire study protocol (e.g., follow-up loss) were excluded from the analysis. In comparison, intention-to-treat analysis included every subject who was randomized and missing data at the follow-up assessment was imputed by using the last observation carried forward approach.

To analyze the differences of functional outcomes between sham and real VR group immediately and 1 month after the last VR session, analysis of covariance (ANCOVA) was used to control the variance in the baseline scores. The baseline score was the covariate, group was the independent variable, and post-test or follow-up scores were the dependent variable. Because only FMA immediately after last VR session was the primary outcome and others were secondary outcomes, an uncorrected 2-tailed P < .05 was considered statistically significant.

Changes of functional outcomes within each group were first analyzed with one-way repeated measures ANOVA for multiple time points. Paired t test with Bonferroni correction was applied as a post-hoc test with only when repeated measures ANOVA revealed overall significant differences (P < .05) and a 2-tailed P < .017 was considered statistically significant.

Cohen kappa coefficient was used to compute the agreement of group allocation between real allocation and subject's guess. All statistical analyses were performed using the PASW statistical package (SPSS version 18.0, SPSS, Chicago, IL).

3. Results

3.1. Subjects

Of the total of 30 eligible patients, 23 were enrolled and allocated to the sham (n = 11) and real (n = 12) VR groups (Fig. 3). During the consecutive intervention period, 1 patient in the real VR group and 2 patients in the sham VR group dropped out. At the 1-month follow-up, 1 patient in the sham VR group was lost to follow-up (Fig. 3). There were no significant differences between the real and sham groups in terms of demographic variables, stroke subtype, hemiplegic side, onset of stroke, and functional outcomes both in per-protocol and intention-to-treat analyses (Table 1).

Figure 3.

Enrollment, randomization, and follow-up of the stud subjects. VR = virtual rehabilitation.

Table 1.

Baseline characteristics of subjects.

3.2. Primary outcome

FMA immediately after last VR session was not significantly different between sham (46.8 ± 16.0) and real VR group (49.4 ± 49.0) (P = .937 in intention to treat analysis) (Table 2). FMA showed significant trends for improvement across time (P < .001) and FMA immediately after the VR session significantly improved compared with the baseline in the intention to treat analysis of both groups (Table 2).

Table 2.

Functional outcomes at baseline, immediately after the last VR session (post 1), and 1 month after the last VR session (post 2).

3.3. Secondary functional outcomes

B-stage for the arm and hand, BBT, and K-MBI showed significant improvements in both real and sham VR group, but the group differences were not significant in all groups (Table 2).

3.4. Total activity counts (TAC) in the hemiparetic upper limb

Table 3 shows the TACs in the hemiparetic arm during each 30-minute session of the 3 types of intervention (control [conventional occupational therapy], sham VR, and real VR). TACs in the hemiplegic upper limb could be used in 29 subjects (control: 12, sham VR: 8, real VR: 9) and the baseline characteristics were not significantly different (Table 3). ANOVA showed significant differences between the groups (F_2,26 = 4.43; P = .22). In the post-hoc analysis, real VR group (107,926 ± 68,874) showed significantly more TACs compared with the sham VR group (46,686 ± 25,814), but there was no statistical significance between the real VR and control groups (64,575 ± 27,533).

Table 3.

Baseline characteristics and total activity counts during therapy in subjects who provided the accelerometer data.

3.5. Direct intervention time during VR by therapists

Direct intervention time data were measured from 11 patients in real VR group. Average direct intervention time was 223 ± 94 seconds during the total 30 minutes of real VR.

3.6. Consistency between the real group allocation and the subject's guess

Three patients dropped out of the questionnaire after the last session of VR session and 5 patients did not choose 1 group despite the assessor's instruction, therefore data from 15 patients were used to calculate Cohen kappa coefficient. Cohen kappa coefficient was 0.167 and was not statistically significant (P = .519).

4. Discussion

This randomized, sham-controlled, double-blind study shows that upper limb function after stoke in both real and sham VR groups improves significantly with time in the subacute stage. The compliance for both real and sham VR groups was good. The reason for drop-out during the VR session, early discharge, was not directly related with the VR (Fig. 3).

VR for upper limb recovery after stroke has been investigated by using commercial gaming system (e.g., Nintento wii [Nintendo, Kyoto, Japan]) or specifically developed system.^[27] Although a recent meta-analysis indicated that VR is significantly effective on the upper limb recovery after stoke, the quality and sample size of each included study were low and the results of individual studies were heterogeneous.^[28] Recent randomized controlled studies with large sample size involving patients with subacute stroke showed no significant beneficial effect of non-immersive VR on upper limb recovery compared with the control group (active control or no therapy),^[14,29] similar with our results. The negative results of VR in this study could have been caused by the following possible limitations. First, although our system was specifically developed to induce task-oriented repetitive motions according to the patient's ability, the induced motion itself might be less relevant or challenging. The motion in the real world and virtual world can be different and transfer of virtual training to the real world may not be possible in some conditions.^[30] Our Kinect-based system can induce the repetitive arm and wrist motion effectively, but the hand motion which requires more fine motor activity and sensory feedback may be related with less relevant tasks compared with the real task.^[20] In addition, although our system provided the various difficulty levels during the rehabilitation, the challenge level may not be sufficient to induce the continuous active participation and neural plasticity.^[31] Lastly, we used the sham VR group as an active control to blind the subjects, because previous studies using VR had not tried double-blind design. The low and non-significant consistency between the real group allocation and the subject's guess demonstrates that subject blinding was successful. Although we expected the sham VR group to show less task-relevant activity and low dose, the combination of repetitive reaching and pushing the button during sham VR can also induced sufficient motion and some plastic changes, which might neutralize the results.

In spite of limitation of this study, the results demonstrate the perspectives of low-cost VR use in subacute rehabilitation. In this study, the direct intervention time by the therapist was 3.7 minutes on average during the 30-minute treatment session. The real VR group showed more activity in the hemiparetic upper limb than the sham VR group, and similar activity compared with the conventional occupational therapy group during the same duration of the rehabilitation sessions (Table 3). This means that it will be more feasible to apply this system as an adjuvant additional rehabilitation modality in the rehabilitation setting, which may be a tool for group therapy with supervision of 1 therapist or non-therapist such as a nurse or self-exercise between the therapy sessions or during weekend, to induce more repetitive movements, more recovery, and favorable outcomes.^[32–34] The many portions of direct intervention time composed of selecting appropriate program according to the patient's recovery. If the automated algorithm can be adopted to this system, the direct intervention time could decrease and the supervision of more patients by one therapist using this system could be available.^[35] In addition, our developed system can be easily used if the personal computer and depth-sensing camera is prepared. Various depth-sensing cameras have been developed and the cost is decreasing,^[36] therefore the cost for setting this depth-sensing camera will also decrease, which may guarantee easier accessibility. In addition, the future development of depth sensing camera in terms of closer distance detection (e.g., Intel's Real sense)^[37] or higher definition with many references^[38–41] can be applied to simplify our camera-based rehabilitation system.

5. Conclusion

The low-cost Kinect-based upper limb rehabilitation system we developed was not more efficacious compared with the active sham VR control in this randomized, controlled, double-blind trial with subacute stroke patients. However, the compliance in VR was good and our VR system induced more arm motion than control and similar activity compared with the conventional therapy. This characteristic of our system suggests the usability as an adjuvant additional therapy during inpatient stroke rehabilitation.

More advances in our current pilot version VR system with more detailed difficulty levels, task-relevant design, are numerous pleasurable gaming options could induce more active participation and transfer to the real environments. Automated adoption system or more user friendly interface to select the appropriate rehabilitation program or difficulty level can reduce the direct intervention time by a therapist and increase the usability by patients or non-experts themselves. Further clinical trials with this low-cost VR system have to be more pragmatic, focusing on the feasibility as a group or self-therapy in the inpatient or home setting and a cost-effectiveness analysis has to be included.

Author contributions

Conceptualization: Won-Seok Kim, Sungmin Cho, Ji-Young Lee, Nam-Jong Paik.

Data curation: Won-Seok Kim, Seo Hyun Park, Ji-Young Lee, SuYeon Kwon.

Formal analysis: Won-Seok Kim.

Funding acquisition: Won-Seok Kim, Nam-Jong Paik.

Investigation: Sungmin Cho, Seo Hyun Park, Ji-Young Lee, SuYeon Kwon, Nam-Jong Paik.

Methodology: Won-Seok Kim, Ji-Young Lee.

Project administration: Seo Hyun Park, Nam-Jong Paik.

Resources: Sungmin Cho, Ji-Young Lee, SuYeon Kwon.

Software: Sungmin Cho.

Supervision: Nam-Jong Paik.

Visualization: Won-Seok Kim, Sungmin Cho, SuYeon Kwon.

Writing – original draft: Won-Seok Kim, Sungmin Cho.

Writing – review and editing: Seo Hyun Park, Ji-Young Lee, SuYeon Kwon, Nam-Jong Paik.