Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jul 1.
Published in final edited form as: Top Stroke Rehabil. 2019 Dec 25;27(5):321–336. doi: 10.1080/10749357.2019.1701178

Feasibility of integrative games and novel therapeutic game controller for telerehabilitation of individuals chronic post-stroke living in the community

Grigore C Burdea 1,2, Namrata Grampurohit 1,3, Nam Kim 1, Kevin Polistico 1, Ashwin Kadaru 1, Simcha Pollack 4, Mooyeon Oh-Park 5,7, AM Barrett 5, Emma Kaplan 5, Jenny Masmela 5, Phalgun Nori 6
PMCID: PMC8130884  NIHMSID: NIHMS1546409  PMID: 31875775

Abstract

Background:

Intensive, adaptable and engaging telerehabilitation is needed to enhance recovery and maximize outcomes. Such services may be provided under early supported discharge, or later for chronic populations. A novel virtual reality game-based telerehabilitation system was designed for individuals post-stroke to enhance their bimanual upper extremity motor function, cognition, and wellbeing.

Objectives:

To evaluate the feasibility of novel therapeutic game controller and telerehabilitation system for home use.

Methods:

Individuals chronic post-stroke and their caregivers were recruited (n=8+8) for this feasibility study. One was a screen failure and seven completed four weeks (20 sessions) of home-based therapy with or without remote monitoring. Standardized clinical outcome measures were taken pre- and post-therapy. Game performance outcomes were sampled at every session, while participant and caregiver subjective evaluations were done weekly.

Results:

There was a 96% rate of compliance to protocol, resulting in an average of 13,000 total arm repetitions/week/participant. Group analysis showed significant (p < 0.05) improvements in grasp strength (effect size [ES] = 0.15), depression (Beck Depression Inventory II, ES = 0.75), and cognition (Neuropsychological Assessment Battery for Executive Function, ES = 0.46). Among the 49 outcome variables, 36 variables (73.5%) improved significantly (p = 0.001, binomial sign test). Technology acceptance was very good with system rating by participants at 3.7/5 and by caregivers at 3.5/5.

Conclusions:

These findings indicate the feasibility and efficacy of the system in providing home-based telerehabilitation. The BrightBrainer system needs to be further evaluated in randomized control trials and with individuals early post-stroke.

Keywords: adaptable gamification, stroke, telerehabilitation, integrative medicine, BrightBrainer Grasp, caregiver

INTRODUCTION

Approximately 795,000 Americans suffer a stroke each year, estimated to become 3.4 million by 2030 [Benjamin et al. 2017]. Stroke survivors present with varying deficits in motor, cognitive and emotive functioning and a majority receive minimal to no care once considered chronic (6 months post [Bernhardt et al, 2017]). Neuroscience has however shown that brain plasticity in the chronic phase persists well beyond end of current customary rehabilitation [Faralli et al., 2013; Alia et al. 2017]. Function can improve in the chronic phase if therapy is task-oriented, intensive, attended and provided over sufficient duration [Lee 2015]. In order to reduce costs, rehabilitation should be integrative (motor, cognitive, emotive) and done at home, to facilitate needed frequency and duration of training [Bramanti et al., 2018].

Telerehabilitation [Burdea et al., 2000; Richmond et al., 2017; Peretti et al., 2017] is an emerging solution, particularly in areas where therapists are scarce [Saywell et al., 2012; Cherry et al., 2017]. Effective telerehabilitation needs to be applicable to a wide range of disabilities caused by stroke and training intensity needs to be appropriate to the patient’s physical and cognitive impairment for maximal results [Hung et al., 2016]. Thus, telerehabilitation needs to provide a customizable treatment and may use artificial intelligence (AI) to do so.

Studies have proposed virtual reality as a modality for telerehabilitation following stroke [Piron et al., 2008]. Game-based therapy has been widely used to boost motivation, increase exercise intensity, and provide objective quantifiable outcomes [Holden 2005]. However, most off-the-shelf games lack adaptability for people with motor impairments [Webster and Celik, 2014; Cheok, et al., 2015]. Unfortunately, if game difficulty is set too high, it can lead to frustration and abandonment by players with disabilities.

Another limitation in current systems is the device that mediates patient’s interaction in the game. For instance, Nintendo Wii complex game controllers challenge stroke survivors with spastic hands [Cheok et al., 2015]. The Microsoft Kinect is limited by its lack of hand fine movement tracking. Thus, Kinect is not usable for measurement of hand grasp strength or finger extension [Webster and Celik, 2014].

The present article describes a feasibility study of the off-the-shelf VIVE controller and novel BrightBrainer Grasp (BBG) therapeutic game controllers, when used in telerehabilitation of individuals chronic post-stroke.

MATERIALS AND METHODS

The primary aim of the study was to evaluate the benefits of BrightBrainer system and its therapeutic game controllers on physical, cognitive, and emotional functions in individuals chronic post-stroke living at home. The secondary aim was to determine technology acceptance by these individuals, whether low- or high-functioning.

The BrightBrainer rehabilitation system was designed to be mobile, self-contained and to provide intensive and integrative rehabilitation through games [Burdea et al. 2018]. All study procedures, except screening, consent and clinical evaluations, were completed in home settings. The study received initial approval from the Western Institutional Review Board for the system developer and from the Kessler Institutional Review Board for the clinical site.

Telerehabilitation setup

The BrightBrainer system had been previously studied in several clinical settings [Burdea et al., 2013, 2015] however this was its first use in the home. Participants completed their entire training sitting at 60”x30” tables covered with low-friction mats (X-ray Pad) to facilitate supported arm movement (Figure 1a).

Figure 1.

Figure 1.

Open in a new tab

BrightBrainer rehabilitation system using gamification: (a) participant chronic post-stroke performing a grasp strength baseline on the BrightBrainer Grasp novel controller at home with caregiver observing ; (b) HTC VIVE controller [Niehorster et al., 2017] ; (c) therapeutic game controller prototype [Burdea et al., 2019]. © Bright Cloud International Corp. Reprinted by permission.

BrightBrainer Grasp game controller

Participants used either commercially available HTC VIVE controllers [Niehorster et al., 2017] (Figure 1b), or novel BBG controllers (Figure 1c) [Burdea et al. 2019]. This prototype had a rubber pear to detect grasp strength and a curved lever to measure mass finger extension. The rubber pear conformed to spastic hands ensuring good ergonomics. The BBG low-friction curved underside facilitated supported arm movement, including pronation/supination. Wireless communication with the BrightBrainer computer rendering custom therapeutic games ensured no tethers impeded movement. The BBG controller weigh was not an impediment to higher-functioning participants able to lift arms off the table. More design and usability evaluation details are given in [Burdea et al. 2019].

Custom rehabilitation games

Integrative custom games were developed to allow training motor and cognitive function, while benefiting well-being (a sample is shown in Figure 2). Breakout 3D asked participants to destroy an array of crates arranged on an island by repeated shoulder abduction/adductions or flexion/extensions (depending on crates arrangement). This game also trained executive functions (anticipation), hand-eye coordination, and task sequencing. A divider ensured both arms were used during play. Card Island (Figure 2b) trained short term visual and auditory memory in a card pairing game. Towers of Hanoi (Figure 2c) trained arm reaching as well as executive function by asking participants to restack disks in a minimal number of moves, with larger disks not being allowed to be placed on top of smaller ones. Finally, Car Race trained pronation/supination to steer a race car, as well as hand-eye coordination and executive function to avoid obstacles.

Figure 2.

Figure 2.

Open in a new tab

Screen sequences for a sample of BrightBrainer therapeutic games used in the telerehabilitation post-stroke feasibility study: a) Breakout 3D, b) Card Island, c) Towers of Hanoi, d) Car Race. © Bright Cloud International Corp. Reprinted by permission.

All games had versions for the VIVE (3D arm tracking, finger flexion) and for the BBG controller (3D arm tracking, grasping and finger extension). All games adapted to participant’s level of upper extremity (UE) function through baselining. This expanded asymmetrical and limited reach in the physical world to symmetrical and full reach in the virtual environment. Baselining for the VIVE meant computer measurements of reach in horizontal and vertical planes, pronation/supination angles, and index flexion range. For the BBG controller, similar measures were done for arm reach, pronation and supination, to which were added baselines for grasp strength and finger extension.

All games could be played with the more impaired arm only or bimanually, with or without grasping/finger flexion. Congratulatory feedback was to be given at the end of each game, had the participant succeeded. If a game timed out, or a participant did not succeed, an encouraging message followed. Unlike off-the shelf games, participants were never to be told they failed, so to maximize encouragement, motivation, and a feeling of being in control.

Remote therapy training setup

Several BrightBrainer systems were prepared so multiple participants could train simultaneously at different locations. Participant’s homes were inspected first for internet quality and optimal equipment placement. During inspection visits participants and their caregivers were briefed on training procedures and given access to online tutorial videos. These videos showed the proper use of the system, baselining procedures, and ways to play each game. Tutorials were also available during actual training (through a “help” button).

Caregivers were asked to log the number of times they had to assist their participant or any system malfunctions, along with blood pressure and pulse. Households were provided with a blood pressure meter with instructions that readings were to be done pre-, mid-, and post- every telerehabilitation session.

Protocol design

This feasibility study followed an ABA protocol, with data being gathered pre- (A), during (B) and post-training (A). STROBE guidelines for a cohort study were followed for reporting. The Protocol flowchart diagram is shown in Figure 3.

Figure 3.

Figure 3.

Open in a new tab

Protocol flowchart diagram of the BrightBrainer telerehabilitation feasibility study. © Bright Cloud International Corp. Reprinted by permission.

Training lasted 4 weeks, with 5 sessions per week, game difficulty settings being set to change from week to week. An AI module was to change the present game difficulty based on participant’s prior performance playing that game. The rationale for adapting game difficulty based on participant’s past performance stemmed from the need to maintain engagement (i.e. desire to exercise) while at the same time eliciting maximal effort. This is grounded in the “shaping” principle [Taub and Wolf, 1997] which is a golden standard of rehabilitation. If the participant was to succeed three times in a row at a given difficulty, the next time that game was to be played, difficulty was to increase by one level. Conversely, being unsuccessful two times in a row was to result in an automatic reduction in difficulty by one level.

Increasing game difficulty was also to change control modality. For example, in the Breakout 3D game (Figure 2a) higher difficulty levels were to ask participants to remember to press the trigger of the VIVE controller or grasp the BBG controller rubber pear so to bounce a ball with their paddle avatar, lest the ball was lost. This design feature required more concentration and corresponding increase in cognitive load.

To combat boredom, new games were to be gradually added, such that games available for play increased from 4 in week 1 to 8 in week 4. A further change in game control modality related to object selection and number of controllers used. Index finger flexion or grasping were to be a requirement for avatar selection in weeks 2-4, during which period training were to change to bimanual play mediated by two controllers.

A cloud database storage and portal were created for remote access to participants’ game performance data as a way to verify compliance to protocol. This asynchronous manual monitoring was to be supplemented by an AI agent on the server to constantly check participant's progress status. If data had not been uploaded for 3 consecutive days, then the researchers were to be notified.

At the end of each week, patients and caregivers were to fill subjective evaluation forms rating their experience on the system. These custom forms were developed based on human-computer interface acceptability standards [Ozok, 2008]. When logged over time, this was to provide a measure of the responder’s perceived value of the experience and of technology acceptance.

Data Collection Instruments

Motor function and impairment evaluation instruments:

Active Range of Motion (ROM) of Shoulder, Elbow, Hand (all measured with goniometers), Chedoke Arm and Hand Activity Inventory (CAHAI) [Barreca 2015], which measured independence in bimanual activities of daily living (ADLs), Fugl-Meyer [Fugl-Meyer et al, 1975] for UE function (FMA), Jebsen test of hand function (JHFT) [Jebsen et al., 1969], Grip and Pinch Strength (measured with a Jamar dynamometer and pinch meter, respectively), Shoulder Strength (measured with calibrated wrist weights), and Upper Extremity Functional Index (UEFI) standardized questionnaire [Stratford et al., 2001] for self-reported independence in ADLs.

Emotive and cognitive instruments:

Beck Depression Inventory II (BDI-II) [Salkind 1969] for depression severity, Boston Naming Test (BNT) [Kaplan et al., 1983] for language, Brief Visuospatial Memory Test – Revised (BVMT-R) for memory [Benedict 1997], Hopkins Verbal Learning Test (HVLT) – Form 1 and Form 2 for memory [Brandt 2001], NAB-Attention Test [Stern 2003, Gavett 2011], Trail Making Test-A (TMT-A) for attention and processing [Heaton et al., 1991], Trail Making Test-B (TMT-B) for set shifting [Reitan & Wolfson, 1994], Verbal Fluency Test for language (VFT) [Lezak 1995].

An occupational therapist at the Kessler Foundation facility in West Orange, NJ, blinded to the protocol performed all clinical evaluations before and after the 4-week intervention. Participants were compensated for their local travel expenses to attend evaluation sessions.

Participant’s subjective system evaluation instruments:

Participant and caregiver custom forms (Tables 4, 5) each had 10 questions, rating BrightBrainer technology, and perceived benefit. Rating was done on a 5-point Likert scale (1 – least desirable outcome, 5 most desirable one). These questionnaires were developed based on researchers’ heuristic and expertise in the virtual rehabilitation area, and followed human-computer interaction survey design principles [Ozok, 2008].

Table 4.

Feedback values with group averages per question from participants chronic post-stroke. © Bright Cloud International Corp. Reprinted by permission.

Questions for Participants Participant ID Average for all participants
1 3 5 7 9 11* 15
1. The system was easy to use 3.0 3.8 2.0 2.8 3.3 2.8 4.0 3.1
2. Playing games with both arms was easy 3.8 3.0 2.5 2.8 2.3 2.8 3.8 3.0
3. I had no muscle pain or discomfort 2.0 3.8 3.3 5.0 5.0 3.3 4.0 3.8
4. The instruction given to me were useful 4.3 3.8 3.5 5.0 4.8 3.3 4.0 4.1
5. Playing games improved my stroke symptoms 4.0 3.0 4.3 4.0 4.3 2.8 3.0 3.6
6. I was not bored while exercising 3.5 4.0 4.5 5.0 5.0 3.3 3.5 4.1
7. The length of exercising in a day was appropriate, 2.5 3.3 3.8 4.3 4.3 3.3 4.0 3.6
8. There were few technical problems, 2.0 2.8 1.0 2.3 1.8 2.8 3.0 2.2
9. I would encourage others to use it, 5.0 4.0 4.0 4.8 5.0 3.3 4.0 4.3
10. I liked the system overall. 5.0 4.0 4.0 4.0 5.0 3.3 4.0 4.2
Open in a new tab

Note: Participant 11 provided a single data set at the end of the tele-rehabilitation intervention. All other participants provided data sets weekly.

Table 5.

Feedback values with group averages per question from caregivers. © Bright Cloud International Corp. Reprinted by permission.

Questions for Caregivers Caregiver ID Caregivers’ Average
Feedback Score
2 4 6 8 10 12 16
1. The system setup is appropriate for the home environment 4.0 4.7 3.5 1.3 4.0 3.3 4.0 3.5
2. The computer and software were easy to use 3.8 4.7 2.8 3.0 4.0 3.3 3.3 3.5
3. The hand controllers were easy to get on/off at the start/end of the session 4.0 4.3 2.0 1.3 3.3 3.3 3.8 3.1
4. There were few technical problems using the system 3.0 2.0 1.5 2.7 3.8 3.3 3.5 2.8
5. The person I care for did not appear to experience pain or discomfort while exercising 4.0 3.0 2.8 4.0 4.3 3.3 4.0 3.6
6. The person I care for has improved ability to focus on a task 4.3 3.7 3.5 3.0 3.8 3.3 4.0 3.6
7. The person I care for has improved in verbal responses 4.0 3.0 3.0 3.0 4.5 3.3 4.0 3.5
8. I observe that he/she has an improved ability to follow one-step directions 4.0 3.3 3.0 3.0 4.5 3.3 4.0 3.6
9. The person I care for has an improved ability to participate in activities of daily living (Grooming, Dressing, etc.) 4.0 3.3 3.0 3.0 4.3 3.3 4.0 3.5
10. Is able to share his/her experience with the computer games 4.0 4.7 4.5 3.3 3.8 3.3 4.0 3.9
Open in a new tab

Note: Caregiver 12 provided only one feedback evaluation out of the expected four.

Computer data:

Baseline and game performance data were stored at each session and included in automatically-generated session reports. Game data consisted of total session time, total session exercise time, arm repetitions (for each arm), finger flexion (or grasps) per session (hand-specific), total finger extensions per session (hand specific), training intensity (arm repetitions/minute, finger flexion/minute, finger extensions/minute, grasps/minute), average grasp strength, average game difficulty per session, and training time for each cognitive domain in a session.

Vitals:

Blood pressure and pulse measured with an automatic electrical meter (OMRON BP760).

Participant recruitment and characteristics

Elderly chronic post-stroke individuals living in NJ were recruited together with their caregivers so they could assist during telerehabilitation and provide feedback.

Inclusion Criteria:

Age 47 to 80; diagnosis of stroke that occurred more than 9 months prior; English speakers; UE unilateral motor involvement (FMA score 10 to 50); ability to actively move UE more than 15° for shoulder and elbow flexion/extension; more than 4 months post casting procedures or Botulinum toxin injections; cognitive skills to participate (Montreal Cognitive Assessment (MoCA) [Nasreddine et al 2005] score of 18-30).

Exclusion Criteria:

Younger than 47 or older than 80; severe visual neglect or legally blind; severe hearing loss or deafness; receptive aphasia; uncontrolled hypertension (>190/100 mmHg); severe cognitive delay (MoCA score of 17 or less [Freitas et al, 2013]); non-English speakers; a history of violence or drug abuse or not cooperative with the evaluations pre-study; those not able to produce reliable scores on neuropsychological pre-study assessments because they had not comprehended the test, or had severe speech impairment; those with severe hand spasticity (according to the Modified Ashworth Scale [Bohannon and Smith, 1987]) and/or complete lack of arm movement; those who had received recent nerve block or chemo-denervation to the affected UE within 3 months; those unwilling to suspend UE physical therapy provided outside the study during participation in the experimental therapy; those unwilling to allow home inspections.

Data analysis

Comparisons between pre- and post-treatment outcomes were done using paired t-tests. For non-normal data and absent on a normality assumption, a nonparametric Wilcoxon signed rank test was employed [Wilcoxon, 1945]. Categorical variables were analyzed by Chi-square [Pearson, 1900] or Fisher Exact test for 2x2 [1922]. Trends over time in blood pressure and heart rate were estimated and tested using linear regression. A sample size of 8 participants chronic post-stroke was a sample of convenience. This sample size is typical in feasibility studies of telerehabilitation technologies. Due to this small sample size and the exploratory nature of this study, power was usually insufficient for more advanced techniques. A count of tests showing overall improvement in the cohort was tested for significance using the binomial sign test. Ninety-five percent confidence intervals were calculated for all relevant parameters. P-values less than 0.05 were deemed statistically significant; no multiple-test adjustment to the p-value was done. All analyses were conducted using SAS 9.4 (SAS Institute, Inc., Cary, NC). When available, improvements in individual metrics were benchmarked against the Minimal Clinically Important Difference (MCID) for that measure [Lang et al., 2008].

The Kessler Foundation clinical site recruited and consented 8 participants and their 8 caregivers. Once consented, homes were inspected, the experimental systems delivered, and Session #1 performed in the presence of researchers. Subsequent sessions were done by participants with or without the caregiver present, but without researchers at the home. Training was then monitored remotely, to verify adherence to protocol, gauge progress and assist as needed. When technical difficulties arose, researchers troubleshooted by remote access, or at home. One individual was a screen failure. After last session completion, researchers retrieved the systems and conducted exit interviews. No follow-up evaluations were done after the post-intervention standardized clinical evaluations.

RESULTS

Seven participants completed the entire study protocol. They averaged 64.14 years of age (range 48 to 79), were well-educated (averaged 17 school years), predominantly male (4 male and 3 female), Caucasian, averaged 89 months post-stroke (SD 72 months), presented mostly with right sided weakness, had severe UE involvement as indicated by FMA, cognitive impairments as indicated by MOCA , and used a cane for mobility. The caregivers were on average 60.29 years old (range 27 to 74), predominantly male, and Caucasian. Group demographics and clinical characteristics are shown in Table 1.

Table 1.

Participant demographics data. © Bright Cloud International Corp. Reprinted by permission.

Participant Age
(years)		 M/F Race Time post
stroke onset
(months)		 Education
(years)		 Affected
arm		 Co-Morbidities Medication Baseline
Cognitive
Function
(MOCA
scores out of
30)		 Baseline
Arm
Function
(FMA
scores out of
66)		 Ambulation
(Device
used)		 Caregiver
(Age, M/F,
race, years of
school)
001 66 F Asian 28 21 Left Kidney stones, high blood pressure, high cholesterol and hypothyroidism Levothyroxine, Fluoxetine, Atorvastatin, Clopidogrel, Amlodipine 27 37** Cane 66, M, Caucasian, 16
003 79 M Asian 19 16 Right High blood pressure, Diabetes, High Cholesterol, Glaucoma Synthroid, Sertraline, Cosopt, Janumet, Norvasc, Aggrenox, Tylenol Pottasium Chloride, Pantoprazole, Vitamin D3, Iron, Pravastatin, Travatan 18* 23** Cane 66, F, Asian, 16
005 74 M Caucasian 180 16 Right Prostate cancer, Gall bladder surgery, hernia repair Atorvastatin, Clopidogrel, Oxybutynin, Fish oil, Flaxseed oil, Oxycodone/Acetaminophen, Turmeric curcumin 23* 16** Cane 69, F, Caucasian, 12
007 62 M Caucasian 87 16 Left High blood pressure, high cholesterol Lexapro, Aspirin, Diovan, Lipitor 28 21** Cane (sometimes) 27, M, Caucasian, 16
009 64 F African American 23 17 Right Diabetes, hypertension, high cholesterol Metformin, Basaglar, Lipitor, Aspirin, Lisinopril, Fish oil 29 24** Cane/Bioness 62, M, African American, 12
011 56 F Caucasian 74 14 Right Depression, Some vision loss on left eye Sertraline 24* 23** Brace 58, M, Caucasian, 14
015 48 M Caucasian 212 19 Right AVM, shunt, greenfield filter, Left field cut Lamictal 22* 43** N/A 74, M, Caucasian, 12
Open in a new tab

Note: FMA: Fugl Meyer Assessment; MOCA: Montreal Cognitive Assessment; N/A: Not Applicable

*

MOCA scores ≤ 26 indicates cognitive impairment

**

FMA scores of 43-66 indicate mild impairment; scores of 29-42 indicate moderate impairment; scores of 0-28 indicate severe impairment

Upper extremity functional and impairment outcomes

Table 2 summarizes the motor functional outcomes, UE strength and active range of movement in the affected arm and hand and compares them with the MCID for variables where MCIDs have been published. Pre-training average FMA score (UE subset) was 26.7 (indicative of severe impairment). Post therapy the average FMA score was 29.43, a 2.7 point increase (p=0.11), bringing the average at the lower limit of Moderate Impairment.

Table 2.

Group statistical analysis of motor outcomes following four weeks of integrative telerehabilitation on the BrightBrainer system. Affected side or bimanual activities. © Bright Cloud International Corp. Reprinted by permission.

Variable T1 Mean (SD) T2 Mean (SD) Mean
Difference
T2-T1 		95% CI: T2-T1 Signed
Rank p
value 		MCID and
percentage
who
improved
beyond it-		 Effect Size (CI)**
 
Fugl-Meyer (primary outcome for motor function), affected side 26.71 (9.60) 29.43 (10.72) 2.71 (0.89, 4.54) 0.063 5.2 points 14% 0.27 (−0.79, 1.32)
Chedoke Arm and Hand Function Inventory 23.14 (6.15) 28.00 (6.15) 4.86 (1.53, 8.19) 0.031 6.3 points 43% 0.28 (−0.77, 1.33)
Jebsen Test of Hand Function, affected side (lower scores are better) seconds 1728.55 (1087.29) 1700.51 (1095.77) −28.04* (−58.08, 2.00) 0.125 −20.8 sec 43% 0.03 (−1.02, 1.07)
Upper Extremity Functional Index (UEFI) 26.00 (25.81) 38.14 (25.58) 8.67* (0.31, 17.03) 0.063 8 points 43% 0.42 (−0.64, 1.48)
 
Shoulder strength, (affected Anterior Deltoid) (Newton) 3.82 ( 6.49) 3.82 (6.49) 0.00 N/A N/A N/A N/A
Shoulder strength, (affected Lateral Deltoid) (Newton) 1.60 ( 2.80) 1.60 (2.80) 0.00 N/A N/A N/A N/A
Grasp Strength, affected (Newton) 112.58 ( 92.97) 127.22 ( 90.65) 14.63 (0.80, 28.48) 0.031 N/A N/A
Pinch Strength, affected (Newton) 11.21 (15.66) 15.26 (18.86) 4.05 (−0.13, 8.18) 0.0652 N/A N/A
 
Shoulder Flexion, affected (degrees) 63.29 (33.23) 69.71 (34.53) 6.43 (1.48, 11.38) 0.063 N/A N/A
Shoulder Extension, affected (degrees) 32.57 (9.52) 34.43 (6.50) 1.86 (−2.53, 6.24) 0.500 N/A N/A
Shoulder Abduction, affected (degrees) 57.29 (26.26) 63.00 (32.04) 5.71 (−0.84, 12.27) 0.125 N/A N/A
Shoulder External Rotation, affected (degrees) 37.43 (16.18) 41.71 (11.86) 4.29 (−6.00, 14.57) 0.375 NA NA
Shoulder Internal Rotation, affected 40.57 (21.16) 51.71 (16.25) 11.14* (−2.55, 24.83) 0.094 NA NA
Elbow Flexion, affected 118.14 (25.79) 111.43 (32.24) −6.71 (−43.688, 30.260) 0.875 NA NA
Elbow Pronation, affected 72.86 (15.77) 84.29 (34.09) 11.43* (−17.41, 40.27) 0.625 NA NA
Elbow Supination, affected 56.71 (41.61) 49.29 (37.35) −7.43 (−48.15, 33.30) 1.000 NA NA
Thumb PMP Extension, affected, degrees 10.57 (10.36) 20.71 (25.57) 10.14 (−5.52, 25.81) 0.250 NA NA
Index PMP Extension, affected, degrees 9.14 (10.37) 16.43 (30.10) 7.29 (−14.15, 28.72) 1.000 NA NA
Middle PMP Extension, affected, degrees 6.00 (8.87) 17.14 (24.98) 11.14 (−10.63, 32.91) 0.500 NA NA
Ring PMP Extension, affected, degrees 5.71 (9.76) 15.00 (31.75) 9.29 (−13.44, 32.01) 1.000 NA NA
Little PMP Extension, affected, degrees 7.14 (11.50) 15.00 (26.61) 7.86 (−11.37, 27.08) 1.000 NA NA
Open in a new tab

Note: Bold p values indicate statistical significance or trend statistical significance.

*

indicates improvement larger than the minimally clinically important difference. Bold changes are in the improvement direction, except for Jebsen Hand Function Test. Among the motor evaluation variables, FMA, Chedoke, Jebsen and UEFI showed noticeable improvements. Among the 21 motor outcomes shown, 17 (81%) improved. Affected UE ROM evaluations showed that shoulder flexion, shoulder abduction, shoulder external rotation, elbow flexion, pronation, and supination improved.

The group average task completion time of the Jebsen battery of simulated ADLs was reduced by an average 28 seconds with 43% of participants improving above the MCID of 20.8 seconds [Gomes-Osman & Field-Fote, 2015]. The UEFI score had an 8.7 points average improvement (p=0.045), with 43% of participants improving above the MCID of 8 points [Chesworth et al. 2014].

Affected hand grasping strength increased on average a statistically significant 14.67 N (p=0.041). No change was observed for the group shoulder strength average. Shoulder range of motion improved for all ranges with a large increase of 11.14° in shoulder internal rotation. There was an increase in finger extension range in all affected fingers, with the largest average increase of 11° in middle finger extension (p=0.25), followed by thumb range increase of 10° (p=0.16).

Emotive and Cognitive Outcomes

Table 3 lists group averages for measures of depression severity (Beck Depression Inventory II), Executive Function (NAB and TMT-B), Memory (BVMT-R, NAB, HVLT), attention (TMT-A) and language (BNT and Verbal Fluency Test). As can be seen, average depression severity dropped a statistically significant 3.4 points (p=0.03), however below the MCID of 5 points [Hiroe et al, 2005]. Average values of executive function had a statistically-significant improvement of 3.3 points (p=0.03) as measured by NAB. Of the other 7 cognitive variables, 4 improved.

Table 3.

Group statistical analysis of cognitive outcomes following four week integrative training on the BrightBrainer system. © Bright Cloud International Corp. Reprinted by permission

Variable T1 Mean (SD) T2 Mean (SD) Mean Difference
T2-T1 (95% CI)		 Signed Rank
p value 		MCID and
Percentage of
subjects beyond
it		 Effect Size**
(CI)
Beck Depression Inventory-II (depression) (Lower score is better) 9.57 (5.41) 6.14 (3.53) −3.43 (−7.20, 0.34) 0.0313 5 points 14% 0.75 (−0.33, 1.83)
Brief Visuospatial Memory Test Revised - Trials 1-3 (memory) 5.57 (0.79) 5.43 (1.13) −0.15 (−1.60, 1.31) 1.000 N/A −0.14 (−1.19, 0.91)
Neuropsychological Assessment Battery (executive function) [Primary Outcome] 8.86 (5.43) 12.14 (8.59) 3.29 (0.095, 6.477) 0.031 N/A 0.46 (−0.60, 1.52)
Neuropsychological Assessment Battery (attention) 17.14 (4.06) 20.20 (5.77) 2.86 (−1.72, 7.43) 0.188 N/A 0.61 (−0.46, 1.69)
Hopkins Verbal Learning Test Form 1 & 2, Total # of hits, (memory) 11.00 (1.83) 10.43 (1.62) −0.57 (−1.97, 0.83) 0.500 N/A −0.33 (−1.38, 0.72)
Boston Naming Test 13.17 (2.64) 13.33 (2.25) 0.16 (−0.87, 1.20) 1.000 0% 0.07 (−0.98, 1.11)
Trail Making Test Version A, (attention/processing) (Lower score is better) 41.14 (14.99) 41.14 (19.99) 0.00 (−9.63, 9.63) 1.000 0% N/A
Trail Making Test Version B, (set shifting) (Lower score is better) 151.57 (105.48) 118.57 (71.87) −33.00* (−85.55, 19.55) 0.219 29% 0.37 (−0.69, 1.42)
Verbal Fluency Test 29.57 (19.82) 32.00 (20.17) 2.43 (−2.99, 7.84) 0.438 N/A 0.12 (−0.93, 1.17)
Open in a new tab

Note: Bolded values indicate statistical significance (p < 0.05).

*

Indicates improvement over time greater than the minimal clinically important difference.

**

Effect size interpretation: Negligible, < 0.2, Small, 0.2 to 0.49; Moderate, 0.50 to 0.70; Large, > 0.7 (Bold changes are in the improvement direction. Higher scores are better function for all variables except Beck Depression Inventory and Trail Making Test. Among 9 total variables, 6 variables (66.7%) improved. CI: 95% Confidence Interval; T1: Pre-test scores; T2: Post-test scores; SD: Standard Deviatio

Game performance data

Figure 4 shows group game performance averages and SD for total number of minutes played over 4 weeks, total number of arm repetitions, total number of finger flexions/extensions, and training intensity (total number of repetitions/total minutes played). Over the 4 weeks (20 sessions) of telerehabilitation, participants averaged at total of about 10 hours of therapeutic game play (605 minutes), about 55,000 total arm repetitions and about 6,800 total finger flexions. These values are inflated due to over counting of repetitions (and thus intensity of training) when severely impaired participants used the VIVE controller. When subjects had an extremely small arm reach, shaking the controllers rapidly in any direction gave high repetition counts. The same therapeutic games induced high repetition counts on other populations and with a different game controller (see the Discussion section for details). Thus there is high confidence that the repetitions in the current study well exceed the 400 attended task-related arm repetitions needed to induce brain reorganization [Nudo 1997].

Figure 4.

Figure 4.

Open in a new tab

Game-based variables weekly averages and standard deviations for 7 participants chronic post-stroke training at home: a) Weekly game play hours; b) Number of arm repetitions; c) Number of finger flexions and extensions; d) Arm repetitions per minute. Note that for the first week, only the affected side was used which gives zero values and zero standard deviations. © Bright Cloud International Corp. Reprinted by permission.

Arm repetitions were calculated by the system based on arm reach extremes measured at session baseline. The computer then measured the left and right extremes of this rectangle to determine the distance one repetition would equate to. This technique was used in the vertical (up and down) and frontal (forward and backward) planes. When playing a game, the total distance moved that was measured by the game controllers was divided by the distance of one repetition calculated at that session baseline.

Participants’ subjective evaluation of technology and its benefit

Table 4 shows the 10 statements in the custom subjective evaluation form the participants rated at the end of every week. On average participants responded most positively to the statement “I would encourage others to use it” (rating 4.3/5). This was followed by “I like the system overall” (4.2/5), “The instructions given to me were useful” (4.1/5) and “I was not bored while exercising” (4.1/5). The lowest score was for the statement “There were few technical problems” (2.2/5).

Caregivers’ subjective evaluation of technology and its benefit

Table 5 lists the 10 statements caregivers used to rate the system and its perceived benefits for the participants they cared for. The highest rating was for the statement “Is able to share his/her experience with the computer games.” (3.9/5). The next highest score (3.6/5) was for the statements “The person I care for did not experience pain or discomfort while exercising,” “The person I care for has improved ability to focus,” and “I observe that he/she has an improved ability to follow one-step directions.” The lowest score (2.8/5) was for the statement “There were few technical problems using the system,” similar to the lowest rating given by the participants for the same statement.

DISCUSSION

This study assessed the feasibility and efficacy of the BrightBrainer system used for the first time in home telerehabilitation. While this system and games had been designed to offer rehabilitation training in a self-contained format, a number of challenges to home use emerged. During home installations the BrightBrainer weight became a problem when participants wanted the system installed at a second floor of their home, or if the home itself was accessible by stairs from the street. In future designs weight will need to be reduced to facilitate installation regardless of elevation.

A common issue experienced during home training was communication reliability. In such situations BrightBrainer used an amplified WIFI dongle with AC1200 protocol equipped with external WIFI antenna to strengthen the WIFI signal. In rare cases game data were not stored properly during some trials. Data completeness was then restored through backup paper forms (for vital signs) and through extrapolation from existing data. On very few occasions, the training session software closed unexpectedly. In such cases, engineers modified the game data structure for the game performance data packet to periodically upload to the cloud server rather than uploading at the end of the session.

Overall feedback from participants and caregivers indicated high levels of technology acceptance and patient engagement. Participants and caregivers alike wished the training lasted longer. Both groups gave lowest ratings to the frequency of technical problems. Due to the completion timeline of BBG controller prototypes, the first 5 participants (1, 3, 5, 7, and 9) used HTC VIVE controllers, while the last two (participants 11, 15) used BBG controllers. The VIVE had been designed for able hands, and technical problems ensued when used with spastic hands, in part due to the difficulty of holding them in the affected hand. Had the BBG controller been available for use by all participants, it is believed that fewer technology problems would have been encountered. Furthermore, game performance would have been more uniform, with lower STD than those shown in Figure 4.

One problem with the BBG controllers were leakages from the squeezed pressure bulb. Due to the intricate wire routing within its compact design, sealing points were prone to leakage, and an accurate air pressure reading was difficult to obtain. This issue, was resolved by adding a sealant around controller orifices and joints. Augmentative design modifications of the joints were made to bolster structural rigidity of BBG device in view of large torques applied by severely impaired hands which lead to breakage.

The Results section mentioned that there was over-counting of repetitions when severely impaired participants used the VIVE controller. The way arm repetitions were counted introduced errors when subjects had an extremely small arm reach, such that shaking the controllers rapidly in any direction gave high repetition counts. If the distance the game controllers moved was small, dividing it by a small number from the calculated value used to determine one repetition count resulted in over-counting repetitions compared to counting the same movement by a therapist.

Another reason for arm repetitions over count were tracking artifacts during supported arm movement. When a spastic hand had difficulty holding the VIVE controller properly, and it rotated within the hand, tracking degraded and hand avatar jumpiness ensued. It is believed that was the reason of the very high counts for Participant 9 (Table 4) compared to repetitions logged by Participant 15 (the highest functioning in the group). Furthermore, Participant 15 used the BBG controller which had a different tracker from the VIVE one and which was mounted away from the supporting surface. With tracking artifacts minimized, use of BBG controllers resulted in counts of about 15,000 arm repetitions over the approximately 600 minutes (10 hours) of total play over 4 weeks of home training.

The repetition counts per session measured in this study are consistent with values found in [Burdea et al. 2015b] for participants with dementia (but no motor impairments) playing the games described above using Hydra game controllers [Sixsense, 2011]. The group (n=11) averaged up to 1,000 arm repetitions/session corresponding to 25 arm repetitions/min. This is comparable to the 25 arm repetitions/minute for Participant 15 in this study. In a subsequent study on military service members chronic post-TBI or post-stroke, using the same games and Hydra controllers [Burdea et al. 2018], group (n= 21) average was about 1,500 total arm repetition (both arms) per session. These other studies are supportive of the outcomes reported here.

The high training intensity in the current study) may explain some of the clinical benefits of BrightBrainer home use. Another reason may be the adaptable game library which was used by all participants, regardless of game controller type.

Statistically significant improvements in bilateral UE function (CAHAI), and grasp strength (dynamometer), highlight the system benefit in improving bilateral movements and strength, important aspects of higher level functional activities.

Although the affected side values of FMA and JHFT both showed improvements that reached clinical significance (MCID) for JHFT and effect size of 0.27 for FMA, the results were not statistically significant. The improvement in FMA scores was below the MCID of 5.2 points [Wagner et al 2008]. The CAHAI score increase of 4.9 points was also below the 6.3 point MCID [Barreca et al 2005]. These modest improvements in UE function are most probably explained by the large time span post-stroke at which the intervention occurred. Larger improvements may have required a longer training than the 4 weeks of the protocol. This is supported by the findings of the review of VR use in stroke rehab [Laver et al 2017] showing majority of interventions involving more than 10 hours of VR training.

The participants’ UEFI self-report showed a mean improvement that was larger than MCID and a large effect size of 0.40, but not statistically significant. This also points to a limitation in this study of inadequate power to detect the change in FMA, JHFT, and UEFI, and a future powered trial using the effect size estimates obtained in this study is needed. Not uncommon to exploratory studies, the small sample, multiple analyses without corrections, and custom-designed system, also limit generalizability of the findings. Generalizability of findings is further limited by the nature of the study sample, which was predominately low functioning, with Fugl-Meyer UE scores at baseline of 16, 21, 23, 23, 24, 37 and 43 (out of 66 max). This was due in part to the high prevalence of spasticity in the sample (6 out of 7 participants, or 86%, having disabling spasticity). This high prevalence was atypical to the chronic stroke population, where spasticity is found in 20-38% of cases one year or more post stroke [Wissel et al, 2013]. Although generalizability to all stroke patients may be limited, and this approach may be most appropriate for more severely impaired patients, therapies tailored to that vulnerable group are also highly needed.

The finger range increased for all fingers with the largest increase in middle finger extension of 11.14°. This is a remarkable finding considering that, to researchers’ knowledge, there are currently no rehabilitation systems that improve finger extension as part of an integrative telerehabilitation protocol.

Rizzo et al [2005] and August [2005] stated that VR telerehabilitation research will require technological sophistication, incorporation of functionality to improve ADLs, user-focused game libraries and session monitoring capabilities. It is believed the research presented here met the above requirements. Some researchers did not find significant improvements using non-immersive VR or custom VR balance training platforms, even though they shared many positive aspects of VR technology such as open space settings and higher patient motivations with very positive user feedback [Yang et al., 2016; O'Neil et. al, 2018]. On the other hand, Putrio [2014] reviewed several studies with VR that showed moderate to strong efficacy in videogame-driven telerehabilitation (VGDT). However, he stated that targeted research is still missing to determine the feasibility of VGDT.

The missing links highlighted by Putrio [2014] such as system usability and data privacy concerns were addressed in the present study from the very outset. The salient findings from this study include significant (p < 0.05) improvement in depression with a large effect size (0.75) in only 4 weeks. This benefit has also been reported in other VR studies in stroke using commercial systems such as Xbox Kinect controllers [Song & Park, 2015), Sony PlayStation 2 [Flynn et al., 2007], and custom designed systems such as BrightArm Duo [House et al., 2017], and RehabMaster [Shin et al., 2014]. However, unlike the present study, none of these other interventions were delivered remotely and the dosage of training in these other studies ranged from 4 [Flynn et al., 2007] to 8 weeks [Shin et al., 2014; Song and Park, 2015].

To conclude, feasibility was established for a novel, integrative, intensive, adaptive, and self-contained BrightBrainer Telerehabilitation system with a game library that utilized HTC VIVE or proprietary BBG wireless controllers. Statistically significant gains in UE motor function, executive function and depression reduction for people with chronic stroke were demonstrated by this four-week telerehabilitation intervention. These results support conducting a follow-up, randomized controlled trial to determine the clinical effectiveness of the BrightBrainer system.

ACKNOWLEDGEMENTS

This research was supported by NIH grant R43AG052290.

Footnotes

Supplementary Information

ClinicalTrials.gov ID: NCT04713384, Unique Protocol ID: BCI-16-001

Protocol and all related documents approved by WIRB.

CONFLICT OF INTEREST STATEMENT/DECLARATION OF INTEREST

Grigore Burdea PhD is owner of Bright Cloud International Corp, FDA listed Medical Device Establishment. BrightBrainer is a Class I exempt medical device, commercially available in US. The BrightBrainer Grasp controller is not commercially available.

REFERENCES

  1. Alia C, Spallett C, Lai S, Panarese A et al. Neuroplastic Changes Following Brain Ischemia and their Contribution to Stroke Recovery: Novel Approaches in Neurorehabilitation. Front Cell Neurosci. 2017; 11: 76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. August K D Bleichenbacher D and Adamovich S. Virtual reality physical therapy: a telerehabilitation tool for hand and finger movement exercise monitoring and motor skills analysis, Proc IEEE 31st Annual Northeast Bioeng Conf, April 2005. [Google Scholar]
  3. Barreca SR, Stratford PW, Lambert CL, et al. Test-retest reliability, validity, and sensitivity of the Chedoke arm and hand activity inventory: a new measure of upper-limb function for survivors of stroke. Arch Phys Med Rehabil. 2005;86:1616–22. [DOI] [PubMed] [Google Scholar]
  4. Barreca SR. Chedoke Arm and Hand Activity Inventory (CAHAI). Retrieved October 26, 2015, from http://www.cahai.ca/manual.html
  5. Bernhardt J, Hayward KS, Kwakkel G, et al. , Agreed definitions and a shared vision for new standards in stroke recovery research: The Stroke Recovery and Rehabilitation Roundtable taskforce, Int J Stroke, 2017, 12(5), 444–450. [DOI] [PubMed] [Google Scholar]
  6. Benjamin EJ, Blaha MJ, Chiuve SE, et al. On behalf of the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics—2017 update: a report from the American Heart Association. Circulation. 2017;135:e229–e445. [Google Scholar]
  7. Bohannon RW, Smith MB. Interrater reliability of a modified Ashworth Scale of muscle spasticity. Phys Ther 1987, 67:206–207. [DOI] [PubMed] [Google Scholar]
  8. Bramanti A, Manuli A, & Calabrò RS. Stroke Telerehabilitation in Sicily: a Cost-Effective Approach to Reduce Disability, Innov Clin Neurosc, 2018;15(1-2), 11–15. [PMC free article] [PubMed] [Google Scholar]
  9. Burdea G, Popescu V, Hentz V, et al. Virtual Reality-based Orthopedic Tele-rehabilitation. IEEE Trans Rehab Eng. 2000; 8(3):429–432. [DOI] [PubMed] [Google Scholar]
  10. Burdea G, Defais C, Wong K, et al. Feasibility study of a new game-based bimanual integrative therapy. Proc 10th Int. Conf Virtual Rehab, Philadelphia, PA. 2013; 101–108. [Google Scholar]
  11. Burdea G, Polistico K, Krishnamoorthy A, et al. A feasibility study of the BrightBrainer™ cognitive therapy system for elderly nursing home residents with dementia. Disab Rehab – Assist Tech. 2015a;10(5):421–432. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Burdea G, Polistico K, Liu R, et al. BrightBrainer feasibility study in a medical adult day program. Proc. Int. Conf. Virtual Rehab, Valencia, Spain. June 2015b, 57–64. [Google Scholar]
  13. Burdea G, Polistico K, Grampurohit N, et al. Concurrent Virtual Rehabilitation of Service Members Post-Acquired Brain Injury – A Randomized Clinical Study, Proc. 12th Int Conf Disabil Virtual Reality Tech. Nottingham, UK, September 2018, 22–31. [Google Scholar]
  14. Burdea G, Kim N, Polistico K, et al. Assistive game controller for artificial intelligence-enhanced telerehabilitation post-stroke. Assistive Tech. June 10, 2019. 1–12 pp. doi: 10.1080/10400435.2019.1593260 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Chesworth BM, Hamilton CB, Walton DM et al. Reliability and validity of two versions of the upper extremity functional index. Physiotherapy Canada. Physiothérapie Canada. 2014;66(3):243–253. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Cherry CO, Chumbler NR, Richards K, et al. Expanding stroke telerehabilitation services to rural veterans: a qualitative study on patient experiences using the robotic stroke therapy delivery and monitoring system program. Disab Rehab. Assist Tech, 2017;12(1), 21–27. 10.3109/17483107.2015.1061613 [DOI] [PubMed] [Google Scholar]
  17. Cheok G, Tan D, Low A, et al. Is Nintendo Wii an Effective Intervention for Individuals With Stroke? A Systematic Review and Meta-Analysis. J A Med Dir Assoc, 2015; 16(11), 923–932. 10.1016/j.jamda.2015.06.010 [DOI] [PubMed] [Google Scholar]
  18. Cherry CO, Chumbler NR, Richards K, et al. Expanding stroke telerehabilitation services to rural veterans: a qualitative study on patient experiences using the robotic stroke therapy delivery and monitoring system program. Disab Rehab. Assist Tech, 2017;12(1), 21–27. 10.3109/17483107.2015.1061613 [DOI] [PubMed] [Google Scholar]
  19. Faralli A, Bigoni M, Mauro A, et al. Noninvasive strategies to promote functional recovery after stroke. Neural Plast. 2013;2013:854597. doi: 10.1155/2013/854597. Epub 2013 Jun 24. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Fisher RA. On the interpretation of χ2 from contingency tables, and the calculation of P". J Royal Stat Soc. 1922. 85 (1): 87–94. doi: 10.2307/2340521. JSTOR 2340521. [DOI] [Google Scholar]
  21. Flynn S, Palma P, Bender A. Feasibility of using the Sony PlayStation 2 gaming platform for an individual poststroke: a case report. J Neurol Phys Ther. 2007. December;31(4):180–9. doi: 10.1097/NPT.0b013e31815d00d5. [DOI] [PubMed] [Google Scholar]
  22. Freitas S, Simões MR, Alves L, et al. Montreal cognitive assessment: validation study for mild cognitive impairment and Alzheimer disease. Alzheimer Dis Assoc Disord. 2013. Jan-Mar;27(1):37–43. doi: 10.1097/WAD.0b013e3182420bfe. [DOI] [PubMed] [Google Scholar]
  23. Fugl-Meyer AR, Jaasko L, Leyman I, et al. , The post-stroke hemiplegic patient population: A method for evaluation of physical performance. Scand J Rehabil Med. 1975; 7(13), 13–31. [PubMed] [Google Scholar]
  24. Gavett BE. Neuropsychological Assessment Battery. In: Kreutzer JS, DeLuca J, Caplan B (eds) Encyclopedia of Clinical Neuropsychology. 2011, Springer, New York, NY [Google Scholar]
  25. Gomes-Osman J, & Field-Fote EC. Improvements in hand function in adults with chronic tetraplegia following a multiday 10-Hz repetitive transcranial magnetic stimulation intervention combined with repetitive task practice. J Neuro Phys Ther. 2015;39(1), 23–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Hiroe T, Kojima M, Yamamoto I., et al. Gradations of clinical severity and sensitivity to change assessed with the Beck Depression Inventory-II in Japanese patients with depression. Psychiatry Res. 2005;135(3):229–235. [DOI] [PubMed] [Google Scholar]
  27. Holden M Virtual environments for motor rehabilitation: review. Cyberpsychol Behav. 2005;8(3):187–211 [DOI] [PubMed] [Google Scholar]
  28. Hung YX, Huang PC, Chen KT, et al. What Do Stroke Patients Look for in Game-Based Rehabilitation. Medicine, 2016;95(11). 10.1097/MD.0000000000003032 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Jebsen RH, Taylor N, Trieschmann RB, et al. An objective and standardized test of hand function. Arch Phys Med Rehabil. 1969; 50, 311–319. [PubMed] [Google Scholar]
  30. Kaplan E, Goodglass H, Weintraub S, et al. Boston naming test. Philadelphia: Lea & Febiger, 1983. [Google Scholar]
  31. Lang CE, Edwards DF, Birkenmeier RL, et al. Estimating minimal clinically important differences of upper-extremity measures early after stroke. Arch Phys Med Rehabil. 2008. September;89(9):1693–700. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Lee KB, Lim SH, Kim KH, et al. Six-month functional recovery of stroke patients: a multi-time-point study. I J Rehab Res. 2015;38(2), 173–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Niehorster DC, Li L, & Lappe M. The Accuracy and Precision of Position and Orientation Tracking in the HTC Vive Virtual Reality System for Scientific Research. i-Perception, 2017; 8(3), 2041669517708205. doi: 10.1177/2041669517708205 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. O'Neil O, Fernandez MM, Herzog J, et al. Virtual Reality for Neurorehabilitation: Insights From 3 European Clinics, PM and R, 2018;10(9), pp.S198–S206. [DOI] [PubMed] [Google Scholar]
  35. Ozok A, Survey Design and Implementation in HCI. Ch. 58, The Human Computer Interaction Handbook, Sears, Jacko and Jacko, Eds., Taylor and Francis, 2008, 1151–1169. [Google Scholar]
  36. Pearson K, On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling, Philosophical Mag. 1900, Series 5. 50 (302): 157–175. doi: 10.1080/14786440009463897 [DOI] [Google Scholar]
  37. Peretti A, Amenta F, Tayebati SK, et al. Telerehabilitation: Review of the State-of-the-Art and Areas of Application. JMIR Rehabil Assist Technol. 2017;4(2):e7. Published 2017 July 21. doi: 10.2196/rehab.7511 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Piron L, Turolla A, Tonin P, et al. Satisfaction with care in post-stroke patients undergoing a telerehabilitation programme at home. J Telemed Telecare, 2008;14(5), 257–260. 10.1258/jtt.2008.080304 [DOI] [PubMed] [Google Scholar]
  39. Putrio D. Telerehabilitation and emerging virtual reality approaches to stroke rehabilitation, Curr Opin Neurol. 2014. December;27(6):631–6 [DOI] [PubMed] [Google Scholar]
  40. Reitan R and Wolfson D. A Selective and Critical Review of Neuropsychological Deficits and The Frontal Lobes. Neuropsychology review. 1994; 4:161–98. doi: 10.1007/BF01874891 [DOI] [PubMed] [Google Scholar]
  41. Richmond T, Peterson C, Cason J, et al. American Telemedicine Association’s Principles for Delivering Telerehabilitation Services. Int J Telerehab, 2017;9(2), 63–68. 10.5195/ijt.2017.6232 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Rizzo A, Strickland D, Bouchard S. The Challenge of Using Virtual Reality in Telerehabilitation, Telemed J e-Health, 2005;10 (2), 10.1089/tmj.2004.10.184 [DOI] [PubMed] [Google Scholar]
  43. Salkind MR. Beck depression inventory in general practice. J Royal College of General Practitioners, 1969;18(88), 267–71. [PMC free article] [PubMed] [Google Scholar]
  44. Saywell N, Vandal AC, Brown P, et al. Telerehabilitation to improve outcomes for people with stroke: study protocol for a randomized controlled trial. Trials, 13, 2012;233. 10.1186/1745-6215-13-233 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Sixense Entertainment. [Last accessed November 9, 2019] Razer Hydra Master Guide. 2011. Available from: http://drivers.razersupport.com/index.php?_m=downloads&_a=viewdownload&downloaditemid=623&nav=
  46. Song GB, Park EC. Effect of virtual reality games on stroke patients' balance, gait, depression, and interpersonal relationships. J Phys Ther Sci. 2015. July;27(7):2057–60. doi: 10.1589/jpts.27.2057.Epub2015Jul22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Taub E and Wolf SL. Constraint induced movement techniques to facilitate upper extremity use in stroke patients. Top Stroke Rehabil. 1997;3:38–61. https://www.ncbi.nlm.nih.gov/pubmed/27620374 [DOI] [PubMed] [Google Scholar]
  48. Wagner JM, Rhodes JA, et al. Reproducibility and minimal detectable change of three-dimensional kinematic analysis of reaching tasks in people with hemiparesis after stroke. Phys Ther. 2008;88(5):652–663. [DOI] [PubMed] [Google Scholar]
  49. Webster D, & Celik O. Systematic review of Kinect applications in elderly care and stroke rehabilitation. J Neuroeng Rehab, 2014;11, 108. 10.1186/1743-0003-11-108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wissel J, Manack A, Brainin M. Toward an epidemiology of poststroke Spasticity. Neurology. 2013;80 (Suppl 2):S13–S19. [DOI] [PubMed] [Google Scholar]
  51. Wilcoxon F Individual comparisons by ranking methods. Biometrics Bulletin. 1945, 1 (6): 80–83. doi: 10.2307/3001968. JSTOR 3001968. [DOI] [Google Scholar]
  52. Yang Wang HK, Wu RM, et al. , Home-based virtual reality balance training and conventional balance training in Parkinson's disease: A randomized controlled trial, J Formosan Med Assoc, 2016; (115)9, pp734–74. [DOI] [PubMed] [Google Scholar]

RESOURCES