Effect of Specific Over Nonspecific VR-Based Rehabilitation on Poststroke Motor Recovery: A Systematic Meta-analysis

Abstract

Background. Despite the rise of virtual reality (VR)-based interventions in stroke rehabilitation over the past decade, no consensus has been reached on its efficacy. This ostensibly puzzling outcome might not be that surprising given that VR is intrinsically neutral to its use—that is, an intervention is effective because of its ability to mobilize recovery mechanisms, not its technology. As VR systems specifically built for rehabilitation might capitalize better on the advantages of technology to implement neuroscientifically grounded protocols, they might be more effective than those designed for recreational gaming. Objective. We evaluate the efficacy of specific VR (SVR) and nonspecific VR (NSVR) systems for rehabilitating upper-limb function and activity after stroke. Methods. We conducted a systematic search for randomized controlled trials with adult stroke patients to analyze the effect of SVR or NSVR systems versus conventional therapy (CT). Results. We identified 30 studies including 1473 patients. SVR showed a significant impact on body function (standardized mean difference [SMD] = 0.23; 95% CI = 0.10 to 0.36; P = .0007) versus CT, whereas NSVR did not (SMD = 0.16; 95% CI = −0.14 to 0.47; P = .30). This result was replicated in activity measures. Conclusions. Our results suggest that SVR systems are more beneficial than CT for upper-limb recovery, whereas NSVR systems are not. Additionally, we identified 6 principles of neurorehabilitation that are shared across SVR systems and are possibly responsible for their positive effect. These findings may disambiguate the contradictory results found in the current literature.

Introduction

Better medical treatments in the acute phase after stroke have increased survival and with that the number of patients needing rehabilitation with an associated increased burden on the health care system.¹ Novel technologies have sought to meet this increased rehabilitation demand and to potentially allow patients to continue rehabilitation at home after they leave the hospital.² Also, technology has the potential to gather massive and detailed data (eg, kinematic and performance data) that might be useful in understanding recovery after stroke better, improving the quality of diagnostic tools and developing more successful treatment approaches.³ Given these promises, several studies and meta-analyses have evaluated the effectiveness of technologies that use virtual reality (VR) in stroke rehabilitation. In a first review, Crosbie et al⁴ analyzed 6 studies that used VR to provide upper-limb rehabilitation. Although they found a positive effect, they concluded that the evidence was only weak to moderate given the low quality of the research. A later meta-analysis analyzing 5 randomized controlled trials (RCTs) and 7 observational studies suggested a positive effect on a patient’s upper-limb function after training.⁵ Another meta-analysis of 26 studies by Lohse et al,⁶ which compared specific VR (SVR) systems with commercial VR games, found a significant benefit for SVR systems as compared with conventional therapy (CT) in both body function and activity but not between the 2 types of systems. This study, however, included a variety of systems that would treat upper-limb, lower-limb, and cognitive deficits. Saywell et al⁷ analyzed 30 “play-based” interventions, such as VR systems including commercial gaming consoles, rehabilitation tools, and robot-assisted systems. They found a significant effect of play-based versus control interventions in dose-matched studies in the Fugl-Meyer Assessment of the Upper Extremity (FM-UE).⁷ In contrast, a more recent large-scale analysis of a study with Nintendo Wii–based video games, including 121 patients concluded that recreational activities are as effective as VR.⁸ A later review evaluated 22 randomized and quasi–randomized controlled studies and concluded that there is no evidence that the use of VR and interactive video gaming is more beneficial in improving arm function than CT.⁹ In all, 31% of the included studies tested nonspecific VR (NSVR) systems (Nintendo Wii, Microsoft Xbox Kinect, Sony PlayStation EyeToy). Hence, although VR-based interventions have been in use for almost 2 decades, their benefit for functional recovery, especially for the upper limb, remains unknown. Possibly, these contradictory results indicate that, at present, studies are too few or too small and/or the recruited participants too variable to be conclusive.¹⁰ However, alternative conclusions can be drawn. First, VR is an umbrella term. Studies comparing its impact often include heterogeneous systems or technologies, customized or noncustomized for stroke treatment, addressing a broad range of disabilities. However, effectiveness can only be investigated if similar systems that rehabilitate the same impairment are contrasted. This has been achieved by meta-analyses that investigated VR-based interventions for the lower limb, concluding that VR systems are more effective in improving balance or gait than CT.¹¹ Second, a clear understanding of the “active ingredients”³ that should make VR interventions effective in promoting recovery is missing. Therapeutic advantages of VR identified in current meta-analyses are that it might apply principles relevant to neuroplasticity,^5,9 such as providing goal-oriented tasks,^5,9 increasing repetition and dosage,^5,9 providing therapists and patients with additional feedback,^5,6,9 and allowing to adjust task difficulty.⁶ In addition, it has been suggested that the use of VR increases patient motivation,⁶ enjoyment,^8,9 and engagement⁷; makes intensive task-relevant training more interesting^4,7; and offers enriched environments.⁹ Although motivational aspects are important in the rehabilitation process because they possibly increase adherence,³ their contribution to recovery is difficult to quantify because it relies on patients’ subjective evaluation.^7,12-15 Rehabilitation methods, whether VR or not, however, need to be objectively beneficial in increasing the patient’s functional ability. Hence, an enormous effort has been expended to identify principles of neurorehabilitation that enhance motor learning and recovery.^16-24 Consequently, an effective VR system should besides be motivating, also augment CT by applying these principles in the design.²³ Following this argument, we advance the hypothesis that custom-made VR rehabilitation systems might have incorporated these principles, unlike off-the-shelf VR tools, because they were created for recreational purposes. Combining the effects of both approaches in one analysis might, thus, mask their real impact on recovery. Again, in the rehabilitation of the lower limb, this effect has been observed. Two meta-analyses investigating the effect of using commercial VR systems for gait and balance training did not find a superior effect, which contradicts the conclusions of the other systematic reviews.¹¹ In upper-limb rehabilitation, this question has not been properly addressed until the most recent review by Aminov et al.²⁵ However, there are several flaws in the method applied that could invalidate the results they found. Specifically, studies were included regardless of their quality, and it is not clear which outcome measurements were taken for the analysis according to the World Health Organization’s International Classification of Function, Disability, and Health (ICF-WHO).²⁶ In addition, a specifically designed rehabilitation system (Interactive Rehabilitation Exercise [IREX])²⁷ was misclassified as an off-the-shelf VR tool. Because their search concluded in June 2017, the more recent evidence is missing. We decided to address these issues by conducting a well-controlled meta-analysis that focuses only on RCTs that use VR technologies for the recovery of the upper limb after stroke. We analyze the effect of VR systems specifically built for rehabilitation (ie, SVR systems) and off-the-shelf systems (ie, NSVR commercial systems) against CT according to the ICF-WHO categories. Also, we extracted 11 principles of motor learning and recovery from established literature that could act as “active ingredients” in the protocols of effective VR systems. Through a content analysis, we identified which principles are present in the included studies and compared their presence between SVR and NSVR systems. We hypothesized, first, that SVR systems might be more effective than NSVR systems as compared with CT in the recovery of upper-limb movement and, second, that this superior effect might be a result of the specific principles included in SVR systems.

Methods

This meta-analysis was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines.²⁸

Identification of RCTs

We define VR as a computer-based technology that provides the user with a sense of presence in a virtual environment,²⁹ which is induced by exposing the user to computer-generated sources of sensory stimulation that satisfy their perceptual predictions and expected sensorimotor contingencies.³⁰ The studies included aimed at training the upper extremity of stroke patients through active participation, without assistive robotic devices (eg, exoskeleton, end-effector devices) or exogenous stimulation. We compared the impact on body function and activity of 2 kinds of VR systems with CT: SVR and NSVR systems. SVR systems were developed exclusively for neurorehabilitation purposes. NSVR systems, on the other hand, are recreational and/or off-the-shelf video games (eg, Nintendo Wii, Microsoft Xbox). As CT, we considered occupational therapy and physical therapy. To identify all RCTs in these 2 categories, we performed a computerized search in the bibliographic databases MEDLINE (OVID), Cochrane Library Plus (including EMBASE), CINAHL, APA PsycNET, DARE, and PEDro for studies that were published in English from inception until August 7, 2018, the day of the conclusion of the search. The search strategy (Supplementary Table 1) included only RCTs that tested the efficacy of SVR or NSVR systems in recovering the upper limbs of stroke patients who were either in the acute (up to 21 days poststroke), subacute (between 3 weeks and 3 months poststroke), or chronic (after 3 months poststroke) stage. We combined the effects of various chronicity bands because the current literature suggests that principles of motor learning interact constantly with the biological processes of recovery,³¹ and therefore, no differential effect between SVR and NSVR systems resulting from chronicity should be expected. This notion has also been confirmed by the latest meta-analysis.²⁵ In addition, splitting the identified literature into VR type, ICF-WHO category, and chronicity reduces statistical power because of the small number of studies remaining in each band. Two reviewers (BRB and MM) assessed the studies for eligibility. We excluded studies that were not carried out on humans, lacked a control group, included less than 5 participants per experimental condition, did not target upper-extremity rehabilitation, used exoskeletons as interfaces, used exogenous stimulation (such as transcranial stimulation), or did not provide information on standard clinical scales (Figure 1). Exoskeletons and exogenous stimulation protocol where excluded for the passive or active support provided in the rehabilitation process that might lead to different outcomes.

Figure 1.

Study flow diagram (PRISMA). The selection process of identified randomized controlled trials.

Abbreviations: NSVR, nonspecific VR; SVR, specific VR; VR, virtual reality.

Outcome Measurements

Two reviewers (BRB and MM) cross-analyzed the content of the included studies and extracted the relevant data into a separate database. In general, published articles were used. If information in the articles was missing, the respective authors were contacted by mail. To classify the impact of VR on upper-extremity function and activity at the end of therapy according to the ICF-WHO framework, we followed the recommendations given by the Stroke Rehabilitation Evidence-Based Review³² and considered the following outcome measurements in the respective order. For body function, we considered the FM-UE,³³ Modified Ashworth Scale, Motricity Index,³⁴ Brunnstrom Motor Recovery Stage,³⁵ and Stroke Impact Scale (SIS, only hand items).³⁶ For activity, we considered the (Modified) Barthel Index,³⁷ the Functional Independence Measure,³⁸ the Action Research Arm Test,³⁹ the Box and Block Test,⁴⁰ and the Wolf Motor Function Test (WMFT).⁴¹ We did not conduct a comparison for the ICF-WHO category participation because of the 4 studies^8,42-44 that had a corresponding outcome measurement (SIS and Medical Outcomes Study Short Form 36 Health Survey⁴⁵), only one classified as SVR intervention.⁴² For each study, we identified 1 measurement in each category and took the absolute score (mean and SD) at the end of the treatment for intervention and control group. When the SD of the mean was not available,^14,44 we requested it from the corresponding authors. When only the median and first/third quartile^42,46,47 or minimum/maximum^48,49 was reported, we estimated the mean and the SD using the method proposed by Wan et al.⁵⁰

Quality Appraisal and Risk-of-Bias Assessment

We used the established PEDro checklist to assess the quality of the RCTs.⁵¹ In this review, we only included RCTs with a PEDro score of 5 or greater, which we considered to be high-quality studies. We then used The Cochrane Collaboration’s Risk of Bias tool to evaluate the methodological quality of the included studies (Supplementary Figures 1 and 2).

Content Analysis of Included Principles of Neurorehabilitation

To see whether SVR and NSVR systems are different according to their therapeutic specifications, 2 reviewers (MM and BR) reviewed the existing literature on principles of motor learning and recovery for neurorehabilitation. We extracted a list of 11 principles that have been shown to be effective for motor recovery because they enhance neural plasticity and, therefore, optimize acquisition, retention, and generalization of motor skills: massed practice (training that is repetitive),^17,19,22-24 dosage (training that is intensive),^17,19,20,22 structured practice (training that is spaced in time),^17,19,52 task-specific practice (skill training that is relevant for activities of daily living [ADL]),^17,19,23 variable practice (training that is randomized and variable),^16,23 multisensory stimulation (training that provides not only visual feedback),^19,23 increasing difficulty (training that is individualized),^22,23 explicit feedback (training that provides knowledge about results),^18,23 implicit feedback (training that delivers implicit task-relevant cues),^18,23 avatar representation (training that is embodied and immersive),^21,23 and promoting the use of the paretic limb (training that counteracts compensation and learned nonuse).^17,19,22 Each principle was then assigned key descriptors. One of us (MM) then performed a qualitative content analysis in the included studies using the key descriptors as an indicator of whether a given principle was present or not (deductive category application). Only if the key descriptors were explicitly explained or mentioned in the text, the principle was defined to be present in the study. In Table 1, we present the 11 principles that were extracted from the literature together with their definitions, their ascribed effect on recovery, and the assigned key descriptors for encoding. We performed a pure content analysis without following up with the authors to examine the reporting pattern of the principles they thought were relevant for their results. Finally, we calculated the presence as a percentage for each principle, separately for SVR and NSVR studies.

Table 1.

Qualitative Content Analysis: Description, Definition, and Effect of Identified Principles and Their Key Descriptors.

Name	Definition	Effect	Key Descriptors
Massed practice	The number of repetitions performed	Small effects on improvement and retention17,19,22,24	- Number of repetitions was counted - Tasks were aimed at increasing number of repetitions of a movement
Dosage	Training of more than 5 hours a week	Can speed up functional recovery17,19,20,22	- Training is more than 60 minutes of therapy per session and week day
Structured practice	Training schedule with frequent and longer breaks	Better retention than massed protocols17,19,52	- Rests were given during during the session
Task-specific practice	Movements performed are relevant for ADL and goal oriented	Learning is maximal if the task trained is specific17,19	- Tasks incorporated movements that are functionally meaningful (reaching, lifting, grasping pronation, supination, pinching, etc) and were goal oriented - Tasks or movements were relevant for ADL
Variable practice	Several tasks that require different movements	Better retention and enhances generalization16	- Training included various tasks that require a variety of movements
Multisensory stimulation	Providing feedback through multiple senses	Restoration of sensorimotor contingencies19	- Besides visual, other types of feedback were provided (auditory, tactile, etc)
Increasing difficulty	Progressively increase the difficulty of the task or the involved movements	Augment task-specific use of the impaired limb22	- Difficulty or complexity of tasks or movement is changing depending on ability, performance, or time
Explicit feedback	Knowledge about results (task success or failure, or movement outcome)	Retain an adapted movement better18	- Providing cues on task completion with regard to success or failure, or movement outcome (trajectory errors, average completion time, or exactness) - Feedback can also be provided through a therapist
Implicit feedback	Knowledge about performance that is obtained from tracking, analyzing, and visualizing kinematic movement data	Reduce the sensorimotor prediction error and promote learning18	- Real-time visualization of arm/hand movement and other kinematic properties (speed, rotations, synergies compensations) - Display of correct trajectory to follow
Avatar representation	Active execution and observation of movement through an avatar	Degree of agency aids learning from sensorimotor prediction error21	- Virtual movement is represented as a human- or body part–like avatar (whole body, arm, or hand)
Promote use of affected limb	Tasks that are forcing or reinforcing the use of the affected arm	Counteracting learned nonuse17,19,22	- Tasks were designed or required to be performed with the paretic limb - Tasks cannot be accomplished by the healthy arm only

Abbreviation: ADL, activities of daily living.

Statistical Analysis

We performed a subgroup analysis using RevMan 5.1. Outcome measures were included in absolute terms as provided by the authors or estimated from raw data. Heterogeneity was assessed using the χ² test, and I² and was considered significant when the probability value of χ² was <.05 or when I² was >40%.⁵³ The pooled treatment effect (inverse variance) was evaluated using random-effect models to avoid a heterogeneity bias.⁵³ Because a direct comparison between the effects of SVR and NSVR on outcomes for body function and activity is not possible, we conducted an indirect comparison in which each VR type was compared with CT at each ICF level through a subgroup analysis. Because SVR and NSVR studies reported the continuous outcomes in different psychometric scales, the standardized mean difference (SMD) and 95% CI to represent the magnitude of the reported improvement were used. It should be noted that the SMD method does not correct for differences in the direction of the scale. Because WMFT is measured in seconds to complete the task (and, therefore, decreases with better performance), its mean value was multiplied by −1 to ensure that all the scales point in the same direction. For all analyses, the statistical significance level was set at P <.05. Risk of publication bias across studies was estimated visually by inspecting the funnel plots. We used GRADEpro to assess the overall quality of the evidence found.

Results

We wanted to assess whether VR-based systems that are purposefully designed for stroke rehabilitation (SVR) render rehabilitation outcomes different from systems that are NSVR. Our prediction is that SVR systems should outperform NSVR because the former are designed around distinct principles for neurorehabilitation, whereas the latter are not.

Study Identification

We identified 1751 articles that matched the search strategy. Ten additional studies were identified through other sources (eg, meta-analyses). Of the 1164 records screened, 30 articles that were published between January 2002 and August 2018 satisfied the inclusion criteria and were included in this review. The study’s characteristics can be found in Table 2; the aim, the selected outcome measurements per ICF-WHO category and the main finding are reported in Table 3. A total of 1137 records and articles were removed, of which 22 were after qualitative full-text analysis (Supplementary Table 2). One of the articles included 3 experimental subgroups,⁵⁴ which were considered as separate trials, resulting in a total of 32 outcomes that were included in the analysis. A total of 22 RCTs qualified as SVR systems* and 8 as NSVR systems.^{8,43,44,68-72} Of the 30 articles included, 13 evaluated motor function at follow-up after a period of no treatment (SVR = 8, NSVR = 5). Interventions were delivered from 2 to 12 weeks (mean SVR = 4.4 weeks, mean NSVR = 4.3 weeks) across all studies. The duration of the rehabilitation sessions varied in SVR studies from 20 to 158.3 mins (mean 23.9 hours total intervention time), and in the NSVR studies from 60 mins to 135 mins (mean 21.9 hours total intervention time). Overall, the most frequently used outcome measure was the FM-UE (SVR = 16, NSVR = 3).

Table 2.

Characteristics of Included Studies.^a

Author	Intervention	n	Age	DSS	Phase	Type of VR	PEDro
SVR studies
Aşkın et al, 201849	VR + CT vs CT;4 × 5 × 60 (60) = 40 hours	18/38	55, 10.4	603.33 (151.33)	Chronic	VR environment on TV and motion tracking through Microsoft Kinect	6
Brunner et al, 201767	VR + CT vs CT + CT; 4 × 4.1 × 51.1 (107.2) = 43.7 hoursb	57/112	62 (32-88)	34.5 (20)	Subacute	VR environment on computer and motion tracking through data gloves	9
da Silva Cameirão et al, 201155	VR + OT vs intensive OT; 12 × 3 × 20 = 12 hoursb	10/19	61.4 (11.6)	13.2 (5.2)	Acute	VR environment on computer and motion tracking through computer vision and data gloves	7
Crosbie et al, 201256	VR vs CT; 3 × 3 × 30-45 = 4.5-6.8 hoursb	9/18	60.3 (10.9)	329 (216)	Chronic	VR environment in head-mounted display and motion tracking through sensors	9
Duff et al, 201242	VR vs PT; 4 × 3 × 60 = 12 hoursb	11/21	68.8 (8.2)	392 (316)	Chronic	Mixed VR environment and motion tracking through computer vision	6
Jang et al, 200557	VR vs passive control;4 × 5 × 60 = 20 hours	5/10	57.1 (4.5)	414 (88)	Chronic	VR environment on screen and motion tracking through a video camera	5
Jo et al, 201258	VR + CT vs CT;4 × 5 × 60 (18) = 26 hours	15/29	63.85 (7.95)	NA	NA	VR environment on screen and motion tracking through a video camera	6
Kiper et al, 201159	VR + CT vs CT;4 × 5 × 60 (60) = 40 hoursb	40/80	64.0 (16.4)	173.4 (106.5)	Chronic	VR environment on screen and motion tracking through video camera	6
Kiper et al, 201460	VR + CT vs CT;4 × 5 × 60 (60) = 40 hoursb	23/44	64.3 (12.6)	127.8 (94.3)	Chronic	VR environment on screen and motion tracking through video camera	7
Kiper et al, 201865	VR + CT vs CT;4 × 5 × 60 (60) = 40 hoursb	68/136	63.9 (14.1)	127.75 (91.25)	Chronic	VR environment on screen and motion tracking through video camera	6
Kottink et al, 201466	VR vs CT; 6 × 3 × 30 = 9 hoursb	8/18	61.85 (10.65)	1196.9 (743.69)	Chronic	VR environment on horizontal screen and motion tracking through webcam	6
Kwon et al, 201261	VR + CT vs CT;4 × 5 × 30 (70) = 33 hours	13/26	57.5 (13.7)	24.3 (18.1)	Subacute	VR environment on screen and motion tracking through video camera	9
Lee et al, 201662	VR + OT vs TV + OT; 6 × 3 × 30 (50) = 24 hoursb	10/18	71.2 (7.2)	504.9 (196.4)	Chronic	Mixed VR environment on computer and motion tracking through video camera	8
Levin et al, 201215	VR vs OT; 3 × 3 × 45 = 6.75b	6/12	58.95 (14.85)	1168 (383.25)	Chronic	VR environment on screen and motion tracking through video camera	6
Piron et al, 20092	VR vs CT; 4 × 5 × 60 = 20 hoursb	18/36	65.2 (7.8)	405 (158)	Chronic	VR environment on computer and motion tracking through sensors	7
Piron et al, 201063	VR vs CT; 4 × 5 × 60 = 20 hoursb	27/47	60.5 (9)	464 (374)	Chronic	VR environment on screen and motion tracking through sensors	8
Shin et al, 201464	VR + OT vs OT;2 × 5 × 20 (20) = 6.6 hours	9/16	49.3 (8.9)	71.9 (36.9)	Subacute	VR environment on screen and motion tracking trough depth sensor	8
Standen et al, 201648	VR vs passive control;8 × 5 × 60 = 40 hours (maximum), actual ~7 hours	9/18	61 (13.1)	119 (83–279)	Subacute	VR environment on screen and motion tracking trough light-emitting diodes	5
Turolla et al, 201354	VR + CT vs CT;4 × 5 × 60 (60) = 40 hoursb	68/100	62.8 (13.4)	<91	Subacute	VR environment on screen and motion tracking through sensors	5
Turolla et al, 201354	VR + CT vs CT;4 × 5 × 60 (60) = 40 hours	113/170	62.8 (13.4)	91-365	Subacute	VR environment on screen and motion tracking through sensors	5
Turolla et al, 201354	VR + CT vs CT;4 × 5 × 60 (60) = 40 hours	82/106	62.8 (13.4)	>365	Chronic	VR environment on screen and motion tracking through sensors	5
Yin et al, 201446	VR + PT/OT vs PT/OT; 2 × 4.5 × 30 (90) = 18 hours	11/23	58.3 (13.5)	16.3 (7.4)	Acute	VR environment on screen and motion tracking through hand-held sensors	6
Zondervan et al, 201614	VR vs standard at home training; 3 × 3 × 60 = 9 hoursb	9/17	59.5 (40-74)	1551.3 (1058.5)	Chronic	VR environment on laptop and motion tracking through sensors	8
Zucconi et al, 201147	VR vs PT; 4 × 5 × 60 = 20 hoursb	11/22	62.25 (56-73)	236.5 (88-544)	Chronic	VR environment on screen and motion tracking through sensors	8
Mean	4.4 × 4.4 × 49.7 (55) = 23.9 hours	27.1/43.6	61.4	370.4			6.7
NSVR studies
da Silva Ribeiro et al, 201543	VR vs PT; 2 × 2 × 60 = 4 hoursb	15/30	53.3 (7.4)	1559 (1080)	Chronic	Nintendo Wii	5
Kong et al, 201668	VR + PT/OT vs CT + PT/OT; 3 × 4 × 60 (75) = 27 hoursb	33/67	57.5 (9.8)	13.7 (8.9)	Acute	Nintendo Wii	9
Rand et al, 201769	VR vs standard at home therapy; 5 × 6 × 37.6 = 18.8 hoursb	13/24	62 (8.7)	495.8 (263.1)	Chronic	Microsoft Xbox Kinect or Sony PlayStation EyeToy	7
Saposnik et al, 201044	VR + CT vs recreational therapy + CT; 2 × 4 × 60 (60) = 16 hoursb	9/18	61.3 (13)	24.7 (12.5)	Subacute	Nintendo Wii	5
Saposnik et al, 20168	VR + CT vs recreational therapy + CT; 2 × 5 × 60 (37.3) = 16 hoursb	59/121	62 (12.5)	25.8 (9.5-46.75)	Subacute	Nintendo Wii	6
Sin and Lee, 201370	VR + OT vs OT;6 × 3 × 30 (30) = 18 hours	18/35	73.7 (7.5)	239 (64)	Chronic	Microsoft Xbox Kinect	6
Türkbey et al, 201771	VR + CT vs CT;4 × 5 × 60 (60) = 40 hours	10/19	62 (38-79)	47 (13-125)	Subacute	Microsoft Xbox Kinect	9
Yavuzer et al, 200872	VR + CT vs CT + watching VR; 4 × 5 × 30 (60) = 30 hours	10/20	61.1 (8)	118.7 (70)	Subacute	Sony PlayStation EyeToy	8
Mean	3.5 × 4.3 × 52.5 (57) = 21.9 hours	20.9/41.8	61.6	315.4			6.9

Abbreviations: CT, conventional therapy; DSS, days since stroke; NSVR, nonspecific VR; OT, occupational therapy; PT, physical therapy; SVR, specific VR; VR, virtual reality.

^aIntervention: intervention (VR) versus control group (CT, OT, PT), Weeks × Sessions per week × Minutes (if additional CT was given) = Total amount of intervention in hours; n = Number of patients in intervention/Total number of patients. Age: mean years (SD or range). DSS: mean days (SD or range). Phase: acute, 1 day to 3 weeks; subacute, 3 weeks to 3 months; chronic, more than 3 months after stroke.

^bDose matched between groups.

Table 3.

Aim, Outcome Measurements, Main Finding, and Assigned Principles of Included Studies.^a

Author	Aim	ICF-WHO category			Other Scales	Follow-up	Main Finding	Principles
		BF	AC	PP
SVR studies
Aşkın et al, 201849	Effect of VR on upper-limb recovery	FM-UE	BBT		MAS, BS, MI	No	FM-UE significantly higher for VR than control after treatment	- Dosage - Task-specific practice - Variable practice - Multisensory stimulation
Brunner et al, 201767	Compare effectiveness of VR to CT		FIM		BBT, ARAT, Abilhand, PGIC	3 Months	No significant difference after treatment, both groups improved	- Dosage - Task-specific practice - Variable practice - Increasing difficulty - Implicit feedback
da Silva Cameirão et al, 201155	Clinical impact of VR on recovery time course	FM-UE	BI		MRC, MI CAHAI	24 Weeks	FM-UE significantly higher for VR than control after treatment	- Task-specific practice - Variable practice - Avatar representation - Increasing difficulty - Implicit feedback - Promote use of affected limb
Crosbie et al, 201256	Effectiveness of VR to CT on motor rehabilitation	MI	ARAT			6 Weeks	VR maintained improvement in MI at follow-up	- Variable practice - Increasing difficulty - Promote use of affected limb
Duff et al, 201242	Compare VR and PT	FM-UE	WMFT	SIS	MAL QOM/AOU	No	FM-UE significantly higher for control than VR after treatment	- Variable practice - Multisensory stimulation - Explicit feedback - Implicit feedback - Promote use of affected limb
Jang et al, 200557	Effect of VR on cortical reorganization and motor recovery	FM-UE	BBT		MAL QOM/AOU, MFT	No	FM-UE significantly higher for VR than control after treatment	- Task-specific practice - Variable practice - Increasing difficulty - Avatar representation - Implicit feedback - Promote use of affected limb
Jo et al, 201258	Changes in upper-extremity function and visual perception using VR		WMFT		MVPT	No	No significant difference after treatment, both groups improved significantly in WMFT	- Dosage - Structured practice - Variable practice - Increasing difficulty - Explicit feedback - Promote use of affected limb
Kiper et al, 201159	Impact of VR versus CT on treatment of upper extremity	FM-UE	FIM		MAS	No	FM-UE significantly higher for VR than control after treatment	- Dosage - Task-specific practice - Variable practice - Increasing difficulty - Implicit feedback - Promote use of affected limb
Kiper et al, 201460	Is VR more effective than CT on treatment of upper-limb motor function	FM-UE	FIM			No	FM-UE significantly higher for VR than control after treatment	- Dosage - Variable practice - Increasing difficulty - Implicit feedback - Promote use of affected limb
Kiper et al, 201865	Effectiveness of reinforced feedback in VR vs CT	FM-UE	FIM		NIHSS, ESAS	No	FM-UE significantly higher for VR than control after treatment	- Dosage - Task-specific practice - Variable practice - Multisensory stimulation - Explicit feedback - Implicit feedback - Promote the use of affected limb
Kottink et al, 201466	Compare effect of VR to CT on arm function	FM-UE	ARAT			1 Month	No significant difference after treatment, both groups improved significantly in FM-UE	- Task-specific practice - Increasing difficulty - Explicit feedback - Promote the use of affected limb
Kwon et al, 201261	Impact of VR with CT on upper-extremity function and ADL in acute stage	FM-UE	BI		MFT	No	No significant difference after treatment, both groups improved significantly in FM-UE	- Dosage - Task-specific practice - Variable practice - Avatar representation
Lee et al, 201662	Effect of VR on upper-limb function and muscle strength		BBT		JTHFT, GPT	No	BBT significantly higher for VR than control after treatment	- Structured practice - Variable practice - Implicit feedback - Promote use of affected limb
Levin et al, 201215	Potential of VR to improve upper-limb motor ability	FM-UE	BBT		CSI, RPSS, WMFT, MAL QOM/AOU	1 Month	More patients improved in FM-UE in VR than control	- Task-specific practice - Variable practice - Avatar - representation - Increasing difficulty - Explicit feedback - Promote use of affected limb
Piron et al, 20092	Impact of VR on treating motor deficits	FM-UE			Abilhand, MAS	2 And 3 months	FM-UE significantly higher for VR than control after treatment	- Variable practice - Explicit feedback - Implicit feedback - Promote use of affected limb
Piron et al, 201063	Impact of VR versus CT	FM-UE	FIM			No	FM-UE was systematically lower in control than VR	- Variable practice - Increasing difficulty - Explicit feedback - Implicit feedback - Promote use of affected limb
Shin et al, 201464	Assessment of usability and clinical efficacy of VR	FM-UE	BI		MRC	No	FM-UE higher after treatment but not significant for VR	- Task-specific practice - Variable practice - Avatar representation - Increasing difficulty - Explicit feedback - Implicit feedback - Promote use of affected limb
Standen et al, 201648	Feasibility of home-based VR for arm rehabilitation		WMFT		9 Peg hole, MAL QOM/AOU	No	WMFT grip strength at midpoint significantly higher improvement for VR	- Massed practice - Task-specific practice - Variable practice - Increasing difficulty - Explicit feedback - Promote use of affected limb
Turolla et al, 201354	Effectiveness of VR on restoration of upper-limb function and ADL	FM-UE	FIM			No	FM-UE significantly higher for VR than control after treatment	- Dosage - Task-specific practice - Variable practice - Increasing difficulty - Explicit feedback - Implicit feedback
Yin et al, 201446	Effect of VR on rehabilitation of upper-limb motor performance	FM-UE	FIM		ARAT, MAL QOM/AOU	1 Month	No significant difference between groups in FM-UE	- Dosage - Structured practice - Task-specific practice - Multisensory stimulation - Avatar representation - Explicit feedback - Implicit feedback - Promote use of affected limb
Zondervan et al, 201614	Feasibility and efficacy of VR at patient’s home		ARAT		BBT, MAL QOM/AOU 9 Peg Hole	1 Month	MAL QOM change from baseline significant for VR	- Massed practice - Task-specific practice - Multisensory stimulation - Explicit feedback - Promote use of affected limb
Zucconi et al, 201147	Effect of VR on motor impairment	FM-UE	FIM		MAS, RPS	No	Only VR improved significantly after treatment in FM-UE	- Variable practice - Increasing difficulty - Implicit feedback - Promote use of affected limb
NSVR studies
da Silva Ribeiro et al, 201543	Effect of VR vs CT on sensorimotor function and quality of life	FM-UE		SF-36		No	No significant difference after treatment, both groups improved significantly in FM-UE	- Structured practice - Variable practice - Increasing difficulty
Kong et al, 201668	Efficacy of VR with CT on upper-limb recovery	FM-UE	FIM		ARAT, SIS-UL, VAS	7 And 15 weeks	No significant difference after treatment, both groups improved significantly in FM-UE	- Dosage - Variable practice - Explicit feedback - Promote use of affected limb
Rand et al, 201769	Effectiveness of self-training programs on upper-limb function		ARAT		MAL QOM/AOU, BBT	4 Weeks	No significant difference or improvement in MAL QOM after treatment	- Variable practice - Promote use of affected limb
Saposnik et al, 201044	Efficacy of VR for stroke rehabilitation	SIS grip strength	WMFT	SIS	BBT	4 Weeks	VR had significant improvement in WMFT, but only at follow-up	- Dosage - Variable practice - Multisensory stimulation - Avatar representation - Implicit feedback
Saposnik et al, 20168	Compare safety and efficacy of VR with recreational therapy on motor recovery	SIS grip strength	BI	SIS	WMFT, BBT, FIM, MRS	4 Weeks	No significant difference after treatment, both groups significantly improved in WMFT	- Dosage - Task-specific practice - Variable practice - Promote use of affected limb
Sin and Lee, 201370	Effects of additional VR on upper-extremity function	FM-UE	BBT			No	FM-UE significantly higher for VR than control after treatment	- Task-specific practice - Variable practice - Multisensory stimulation - Explicit feedback - Implicit feedback - Promote use of affected limb
Türkbey et al, 201771	Feasibility and safety of VR on upper-limb recovery	BS	BBT		WMFT, FIM	No	No significant difference after treatment, both groups significantly improved in WMFT	- Dosage - Task-specific practice - Variable practice - Multisensory stimulation - Avatar representation - Promote use of affected limb
Yavuzer et al, 200872	Effect of VR on upper-limb motor recovery	BS	FIM			3 Months	BS UE significantly higher in VR than control after treatment	- Dosage - Task-specific practice - Variable practice - Increasing difficulty - Promote use of affected limb

Abbreviations: AC, Activity; ADL, activities of daily living; AOU, amount of use; ARAT, Action Research Arm Test; BBT, Box and Block Test; BF, body function; BI, Barthel Index; BS, Brunnstrom Motor Recovery Stage; CAHAI, Chedoke Arm and Hand Inventory; CSI, Composite Spaticity Index; CT, conventional therapy; ESAS, Edmonton Symptom Assessment Scale; FIM, Functional Independence Measure; FM-UE, Fugl-Meyer Assessment Upper Extremity; GPT, Grooved Pegboard Test; ICF-WHO, World Health Organization’s International Classification of Function, Disability, and Health; JTHFT, Jepsen-Taylor Hand Function Test; MAL, Motor Activity Log; MAS, Modified Ashworth Scale; MFT, Manual Function Test; MI, Motricity Index; MRC, Medical Research Council Grade; MVPT, Motor-Free Visual Perception Test; NIHSS, National Institutes of Health Stroke Scale; NSVR, nonspecific VR; PGIC, Patient Global Impression; PP, Participation; PT, physical therapy; QOM, quality of movement; RPSS, Performance Reaching Scale for Stroke; SF-36, Short-Form Health Survey; SIS, Stroke Impact Scale; SIS-UL, SIS upper limb items; SVR, specific VR; VAS, Visual Analogue Scale; VR, virtual reality; WMFT, Wolf Motor Function Test.

^aIt explains that BF, AC and PP are the ICF-WHO categories.

Assessment of Risk of Bias

We assessed the methodological quality of the included studies, by analyzing each dimension in the risk-of-bias analysis. The detailed analysis per study and the summary plot can be found in Supplementary Figures 1 and 2, respectively.

Allocation

Random sequence generation was adequately reported by 18 SVR and all NSVR studies. One SVR study⁵⁴ stated that no random allocation was performed, and therefore, also no allocation concealment was applied. In the other studies, allocation concealment was adequately reported by 9 SVR and 4 NSVR studies.

Blinding

A total of 18 SVR and all NSVR studies adequately reported that the outcome assessor was blinded. Because of the nature of the interventions, only a few studies could blind participants and therapists. We evaluated studies at a low risk if either of the 2 groups was blind or if they tried to limit the impact of nonblindness (3 SVR and 2 NSVR). Therefore, the nonblinding of personnel and patients could be a high risk of bias.

Incomplete Outcome Data

In all, 19 SVR and 5 NSVR studies adequately reported how missing data points were handled. Two NSVR studies reported inconsistent information about how the missing data were handled.

Selective Reporting

Except for 1 SVR study, all included studies reported the outcomes for all measurements taken.

Effects of SVR and NSVR Interventions

When analyzing the outcome of the subgroup analysis, SVR studies showed a significant impact on the recovery of the upper-limb function (SMD = 0.23; 95% CI = 0.10 to 0.36; P = .0007) and activity (SMD = 0.31; 95% CI = 0.15 to 0.47; P = .0001) that is superior in comparison to CT (Figures 2 and 3, upper panel). NSVR studies showed no significant effect, neither on body function (SMD = 0.16; 95% CI = −0.14 to 0.47; P = .30) nor on activity (SMD = 0.15; 95% CI = −0.15 to 0.45; P = .33); see Figures 2 and 3, lower panel. No significant heterogeneity was present in any comparison. Also, there were no significant differences between the subgroups, neither in body function (P = .70) nor in activity (P = .36) because the CIs overlapped substantially. According to GRADE (Figure 4), there is moderate confidence in the effect estimates for the results found in SVR studies.

Figure 2.

Forest plot of functional outcomes: SVR versus NSVR studies on upper-limb function as measured by the selected outcome.

Abbreviations: SVR, specific VR; NSVR, nonspecific VR; VR, virtual reality.

Figure 3.

Forest plot of activity outcomes. SVR versus NSVR studies on upper-limb activity as measured by the selected outcome.

Abbreviations: NSVR, nonspecific VR; SVR, specific VR; VR, virtual reality.

Figure 4.

Summary of findings for the main comparisons. The quality of evidence for this review was evaluated using GRADEpro, finding a moderate certainty of the effects observed in SVR studies.

Abbreviations: SVR, specific VR; NSVR, nonspecific VR; VR, virtual reality.

Assessment of Reporting Bias

Funnel plot asymmetry might point to a possible publication bias because of a lack of small studies with nonsignificant or unfavorable results (Supplementary Figure 3). Because of our exclusion criteria, only one study had a small sample size⁵⁷ (n = 10). Together with other smaller studies, it skews the plot slightly to the right. However, other explanations are possible. Many SVR systems have become commercially available to clinics after the treatment effect was confirmed through experiments. It, therefore, cannot be ruled out that the confounding factor of conflict of interest could have biased the result described above. Within the included SVR studies, we identified 3 groups of systems called IREX,^57,58,61 Virtual Reality Rehabilitation System (VRRS),^† and other commercial systems^{14,15,55,66,67} that qualified as commercially available devices for clinics. We then separated the funnel plots by these groups and contrasted them with systems that remained experimental set-ups only (Supplementary Figure 4).^‡ The studies using VRRS are large sized, and therefore, cluster at the top of the effect, both in body function outcomes (SMD = 0.21; 95% CI = 0.06 to 0.36; P = .007) and activity outcomes (SMD = 0.38; 95% CI = 0.19 to 0.56; P < .0001). Neither of the other groups reached significance. Therefore, the presence of a bias resulting from commercialization cannot be confirmed.

Evaluation of Included Principles of Neurorehabilitation

We identified relevant differences between SVR and NSVR studies (Figure 5) with respect to the included principles. In Table 3, the assigned principles for each study can be found and the full data set used for the analysis is provided as a supplemental material. First, the spectrum of the principles that are mentioned in more than 50% of the studies is broader in SVR than NSVR interventions. NSVR studies focused on 3 principles—variable practice,^{8,43,44,68-72} promoting of the use of the paretic limb,^8,68-72 and dosage^{8,44,68,71,72}—that were present in 100%, 75%, and 63% of the studies, respectively. SVR studies did not share 1 specific principle in common, but more than 50% of the studies in this category included the same 6 principles: variable practice (86%),^§ promoting the use of the paretic limb (86%),^‖ implicit feedback (64%),^¶ increasing difficulty (64%),^# task-specific practice (64%),** and explicit feedback (59%).^†† We conducted a follow-up analysis to evaluate the effect of dosage because NSVR studies seem to have more intense intervention regimes. We compared the outcomes of those studies that provided more than 60 minutes of therapy per session per week day. We identified 14 studies, 9 SVR^‡‡ and 5 NSVR,^{8,44,68,71,72} that fulfill this criterion. Comparing this subset of SVR and NSVR studies with their respective controls, we still observe a significant superior impact of SVR studies on body function (SMD = 0.23; 95% CI = 0.07 to 0.38, P = .004; see Supplementary Figure 5, upper panel) and activity (SMD = 0.27; 95% CI = 0.13 to 0.41, P = .0002, see Supplementary Figure 6, upper panel), whereas the total number of hours of intervention (SVR: mean [SD] = 35.6 [8] hours; NSVR: mean [SD] = 25.8 [9.1] hours) and the number of weeks (SVR: mean [SD] = 3.8 [0.6] weeks; NSVR: mean [SD] = 3 [0.9] weeks) were not significantly different.

Figure 5.

Distribution of included principles in SVR versus NSVR studies. Blue indicates SVR and red NSVR studies.

Abbreviations: AR, avatar representation; D, dosage; EF, explicit feedback; ID, increasing difficulty; IF, implicit feedback; MP, massed practice; MS, multisensory stimulation; NSVR, nonspecific VR; PUA, promote the use of the affected limb; SP, structured practice; SVR, specific VR; TSP, task-specific practice; VP, variable practice; VR, virtual reality.

Discussion

The use of VR is increasing in neurorehabilitation. However, so far, it is unclear whether VR is effective in enhancing recovery after stroke. We proposed to distinguish between VR systems specifically built for rehabilitation (SVR) and off-the-shelf recreational VR systems (NSVR), based on the assumption that SVR systems incorporate principles of neurorehabilitation that potentially enhance learning and recovery, whereas NSVR systems do not. Our results demonstrate that SVR systems show a higher impact on recovery, on body function, and on activity than CT and that NSVR systems do not. This is in line with evidence found for the use of VR interventions to train balance and gait¹¹ and the most recent meta-analysis on VR interventions.²⁵ The difference between our results and previous analyses is our focus on rehabilitation tools in VR for enhancing upper-limb function and activity only. Hence, the recategorization in SVR and NSVR systems provides a valid basis for the reinterpretation of effects reported in previous reviews.

We propose that the overall positive effect of SVR protocols is a result of the incorporation of principles of neurorehabilitation that enhance motor learning and recovery. Of the 11 principles identified through the literature, we found 6 to be present in more than 50% of the SVR studies. In NSVR interventions, however, only 3 principles surpassed this level. Variable practice ranked high in both SVR and NSVR studies. In VR systems, variable practice can be easily achieved by including a variety of tasks with different goals, movement requirements, and stimuli²³ to enhance learning¹⁶ and retention.⁷³ In addition, the variety can make repetitive training more engaging and enjoyable for the patient and counteract boredom, which has been associated with low adherence to standard training protocols.^3,7 However, variable practice alone is possibly not sufficient to lead to a noticeable effect on recovery. If applied, the 5 additional principles that were present in SVR systems would generate a VR training that challenges the patient optimally through adaptive difficulty^22,23 while providing information on success (results)^18,23 and optimizing implicit error-based learning (performance)^18,23 through tasks that are relevant for ADL (task specific)^17,19,23 besides promoting the use of the paretic arm.^17,19,22 Only 1 SVR study included all 6 principles⁶⁴ and showed a large positive effect for the experimental group in recovery of body function (SMD = 1.13; 95% CI = 0.04 to 2.21) and activity (SMD = 1.47; 95% CI = 0.32 to 2.62).

However, besides known methodological issues,⁹ we note that many protocols relied on the therapists to individualize the practice to the patient’s needs by selecting the training task or movement requirements or adjusting the difficulty parameters.^§§ This might have biased the outcomes and could compromise the internal and external validity of these studies. Computerized systems have the advantage that every principle could be customized to the patient’s individual ability and necessity automatically.⁷⁴ Whereas NSVR systems are typically not similarly adaptive and accessible for modification, this is a unique opportunity for SVR systems.

The results presented in this study do require further investigation for several reasons. First, it must be noted that the included studies may not have published all the details of their intervention. We, therefore, cannot rule out the possibility that VR systems in this analysis might have incorporated principles that were not detected and reported. To conclusively identify the “active ingredients”³ of effective VR systems, a structured interview with the study authors might be the best approach. We see our analysis, however, as a first attempt to shift awareness from form (VR) to content (principles). Second, we recognize that the content analysis could gain validity if it was performed by an independent rater. However, given the relatively small set of indicators and the availability of the full data set with this article, we believe that this risk is sufficiently mitigated. Furthermore, the number of studies included in the NSVR category is relatively small, and therefore, the nonsignificant effect may be a result of low statistical power. However, individual studies do report sufficient sample sizes. In addition, besides the exclusion of small studies, a source of reporting bias may relate to SVR systems that are commercially available to clinics. However, the system with the largest populations clustered well around the mean effect magnitude and the slight skewness is a result of commercial and noncommercial systems. Hence, a bias resulting from financial interest cannot be confirmed. Another potential limitation of our meta-analysis is the high heterogeneity across studies in terms of intervention protocols (eg, training intensity, type of task, movement patterns addressed, etc) and the measurement tools used (eg, the clinical scales). This also made it impossible to provide proof for the clinical relevance of our finding. Values of clinically important differences are not available for all clinical scales and chronicity bands established. For instance, for FM-UE, clinically important differences are available for the subacute⁷⁵ and chronic⁷⁶ phases but not for the acute phase. Despite these limitations, we are confident about the higher impact of SVR systems on motor recovery because the groups were narrowly defined. Our results may also aid researchers in selecting the appropriate principles that drive the desired outcome and then identify the technology that can best implement and deliver these principles. This could be VR alone or coupled with other technologies (eg, robotics or exogenous stimulation), potentially further enhancing recovery.

Conclusion

Overall, our findings suggest that tailor-made VR systems for neurorehabilitation may be valid tools to deliver effective motor rehabilitation poststroke. Future studies should, therefore, not ask if VR should be used or not. Instead, they should investigate which technology, including VR, is most appropriate to facilitate the implementation of principles of neurorehabilitation in a more effective way than CT. We believe that VR is well suited for rehabilitation because it allows the patient to interact in a safe and ecologically valid environment, where the exposure to sensorimotor contingencies can be controlled and modulated in a goal-oriented and autonomous fashion. In our analysis, the superiority of specific VR systems is associated with the following combination of principles that might possibly lead to a greater effect on recovery: task-specific practice, explicit feedback, increasing difficulty, implicit feedback, variable practice, and mechanisms to promote the use of the paretic limb. We are confident that dedicated VR-based systems are well suited for exploiting these principles, and we expect that future technologies will contribute to an even more advantageous implementation of this set of principles underlying recovery and brain repair.