Skip to main content
PMC home page
Psychologica Belgica logoLink to Psychologica Belgica
. 2021 Jun 3;61(1):145–162. doi: 10.5334/pb.1033

The Use of the Term Virtual Reality in Post-Stroke Rehabilitation: A Scoping Review and Commentary

Hanne Huygelier 1, Emily Mattheus 1, Vero Vanden Abeele 2, Raymond van Ee 1,3,4, Céline R Gillebert 1,5
PMCID: PMC8176935  PMID: 34131490

Abstract

Virtual reality (VR) offers many opportunities for post-stroke rehabilitation. However, “VR” can refer to several types of computer-based rehabilitation systems. Since these systems may impact the feasibility and the efficacy of VR interventions, consistent terminology is important. In this study, we aimed to optimize the terminology for VR-based post-stroke rehabilitation by assessing whether and how review papers on this topic defined VR and what types of mixed reality systems were discussed. In addition, this review can inspire the use of consistent terminology for other researchers working with VR. We assessed the use of the term VR in review papers on post-stroke rehabilitation extracted from Scopus, Web of Science and PubMed. We also developed a taxonomy distinguishing 16 mixed reality systems based on three factors: immersive versus semi-immersive displays, the way in which real and virtual information is mixed, and the main input device. 64% of the included review papers (N = 121) explicitly defined VR and 33% of them described different subtypes of VR, with immersive and non-immersive VR as the most common distinction. The most frequently discussed input devices were motion-capture cameras and handheld devices, while regular 2D monitors were the most frequently mentioned output devices. Our analysis revealed that reviews on post-stroke VR rehabilitation did not or only broadly defined “VR” and did not focus on a specific system. Since the efficacy and feasibility of rehabilitation may depend on the specific system, we propose a new data-driven taxonomy to distinguish different systems, which is expected to facilitate communication amongst researchers and clinicians working with virtual reality.

Keywords: Virtual reality, stroke, neurorehabilitation, rehabilitation, review

Introduction

Stroke is a leading cause of disability worldwide, necessitating effective rehabilitation strategies (Institute for Health Metrics and Evaluation (IHME), 2018). It can have a severe and persistent impact on patients with respect to sensorimotor, cognitive and perceptual functions (Edmans & Lincoln, 1987; Hochstenbach et al., 2005; Kauhanen et al., 2000; McDowd et al., 2003; Nys et al., 2007; Pearce et al., 2016; Sue-Min et al., 2002). In the motor domain, patients can have long-lasting impairments affecting their upper-limbs, gait and balance (Hesse & Werner, 2003; Kao et al., 2014; Langhorne et al., 2009; Raghavan, 2015; Sibley et al., 2009; Verheyden et al., 2008). In the cognitive domain, patients can experience difficulties in language, attention, executive functions, praxis and memory (Douiri et al., 2013; Jaillard et al., 2009; Kauhanen et al., 2000; Makin et al., 2013; McDowd et al., 2003; Nys et al., 2007). Stroke can also lead to apathy and depression (Espárrago et al., 2015; Mayo et al., 2009; Nelson et al., 1993), fatigue (Christensen et al., 2008; Snaphaan et al., 2011) and has a negative impact on activities of daily living (Sue-Min et al., 2002) and return to work (Katie et al., 2009).

Virtual reality offers opportunities for post-stroke rehabilitation

There are many challenges in post-stroke rehabilitation. For instance, research investigating different therapies (i.e., physical, occupational, perceptual training) has consistently shown that functional outcome is better when therapy doses are higher (Lohse et al., 2014; Ong et al., 2012; Parker et al., 2013). However, it is not straightforward to deliver high therapy doses in clinical practice (Bernhardt et al., 2007; Lang et al., 2009; Ong et al., 2012; Otterman et al., 2012). Furthermore, treatment adherence to physical therapy of stroke patients is often limited (Langhorne et al., 2009; Miller et al., 2017; Tiedemann et al., 2012). The latter is especially the case when the therapy is experienced to be boring (Miller et al., 2017). Although most research has focused on occupational and physical rehabilitation, there is also evidence that treatment adherence to cognitive rehabilitation can be problematic (Wentink et al., 2018). Finally, it is challenging to develop ecologically valid tasks for cognitive rehabilitation, which may explain why training effects do not transfer to daily life activities (Rizzo et al., 2004).

Virtual reality (VR) has the opportunity to tackle some of these challenges. For instance, it can provide real-time multisensory feedback, task variation, objective progression and task-oriented repetitive training (Laver et al., 2012; Porras et al., 2018; Rizzo et al., 2004). It can also improve the precision of performance measurements and the standardization of treatment protocols (Bohil et al., 2011; Iosa et al., 2012; Massetti et al., 2018; Rizzo et al., 2004; Rose et al., 2005; Tieri et al., 2018). Furthermore, it can have positive effects on motivation. For example, stroke patients reported less fatigue when using a robotic device to navigate a virtual plane displayed on a regular computer monitor than without this visual feedback (Mirelman et al., 2009).

However, the use of virtual reality in post-stroke rehabilitation also faces several challenges. A first challenge is cybersickness. Previous research has revealed that VR applications that create a high sense of presence in the virtual environment also induce more cybersickness (Weech et al., 2019). Other design characteristics of the VR application (Davis et al., 2015; Porcino et al., 2017; Stanney & Hash, 1998) and of the end-users (Arns & Cerney, 2005) can also impact the prevalence of cybersickness. Thus, although current VR games can be designed in ways that effectively minimizes cybersickness in certain populations (Appel et al., 2020; Huygelier et al., 2019; Plechatá et al., 2019), it remains necessary to assess cybersickness for each VR application and end-user group. Second, the potential of VR to increase treatment dosage in post-stroke rehabilitation depends on the ability of patients to use VR systems independently. Although some research has investigated the safety, user experience and usability of the latest generation of VR systems in stroke patients (Huygelier et al., 2020; Lee et al., 2020; Spreij et al., 2020; Weber et al., 2019), none of these studies addressed the feasibility of independent use by patients. Third, although virtual reality offers many opportunities for rehabilitation, most studies did not address the added value nor cost-effectiveness of VR rehabilitation relative to other therapies.

The use of “virtual reality” in post-stroke rehabilitation

In 1994 Milgram et al. (p. 1) (Milgram & Kishino, 1994) defined a VR environment as “an environment in which the participant-observer is totally immersed in, and able to interact with, a completely synthetic world.” Since then, many authors consider the level of immersion as an objective property of the technological system (Bohil et al., 2011; Milgram & Kishino, 1994; Slater, 2003), which depends on the intensity and fidelity of the sensory stimulation provided by that system (Bohil et al., 2011; Milgram & Kishino, 1994; Slater, 2003). It can be distinguished from presence, which refers to the subjective experience of being in the virtual environment without being aware of the technological mediation (Bohil et al., 2011). More immersive technology has a higher chance to create experiences that result in a high sense of presence, but the immersive nature of the technology does not guarantee a high sense of presence. Indeed, in contrast to immersion being related to the technology, sense of presence can vary between individuals. For instance, one study reported that stroke patients reported less sense of presence than healthy individuals, although they used the same immersive VR system (Borrego et al., 2019). In addition, Milgram distinguished several mixed reality systems based on the extent to which real and virtual information were mixed (Milgram et al., 1995; Milgram & Kishino, 1994). Augmented reality (AR) refers to systems in which virtual information is superimposed over the real environment (e.g., Pokémon Go), while in augmented virtuality (AVR) real world information is superimposed over a virtual world, and in VR all information is virtual. Thus, systems in which a virtual avatar represents movements by the user are considered VR, while systems in which an image of the user is added onto a virtual environment are considered AVR.

Although older definitions of VR emphasized its immersive nature and distinguished VR from other categories of mixed reality (Milgram & Kishino, 1994; Steuer, 1992), “virtual reality” has been ill-defined in the literature on post-stroke rehabilitation (Garrett et al., 2018; Perez-Marcos, 2018; Tieri et al., 2018). “Virtual reality” has been used to refer to several types of computerized rehabilitation, ranging from less immersive systems that display 3D environments on regular 2D monitors to more immersive systems that use head mounted displays (HMD) that offer a near full field of view (Huygelier et al., 2019; Iosa et al., 2012; Perez-Marcos, 2018; Tieri et al., 2018). Furthermore, little attention has been given to the dependence of the efficacy and feasibility of VR rehabilitation on the specific system (Garrett et al., 2018). Some of the advantages of VR for stroke rehabilitation may indeed only be characteristic of immersive VR. For instance, immersive VR may increase motivation for rehabilitation tasks, potentially leading to higher therapy doses (Massetti et al., 2018; Tieri et al., 2018). It can also act as an enriched environment with beneficial effects on neuroplasticity (Laver et al., 2012). Furthermore, it allows one to create ecologically valid activities such as car driving, which may in turn improve transfer of rehabilitation effects to daily life (Rizzo & Kim, 2005; Rizzo et al., 2004). Finally, immersive VR allows one to navigate in 3D space, thereby creating opportunities for the rehabilitation of spatial cognition impairments, such as hemispatial neglect (Aravind & Lamontagne, 2014; Dvorkin et al., 2012; Myers & Bierig, 2000; Pedroli et al., 2015).

In summary, the lack of clarity in the use of the term VR has made it difficult to navigate the literature on post-stroke rehabilitation and infer the efficacy and feasibility of specific rehabilitation systems.

The current study

The purpose of this study was to gain insight in the use of the term “VR” in review papers discussing VR post-stroke rehabilitation and to provide new terminology to describe rehabilitation systems. To this end, we extracted the definitions used for VR and which type of mixed reality system was referred to as VR. In addition, we assessed whether different systems were used for the rehabilitation of specific functional impairments and whether immersive VR was more frequently mentioned over time.

Method

We performed a scoping review following the PRISMA guidelines. PubMed, Web of Science Core collection and Scopus were searched for published reviews on VR post-stroke rehabilitation. Publications from the inception of the databases until and including the 29th of July 2019 were searched. The keywords were “stroke”, “virtual reality” and “rehabilitation” and synonyms for these terms (Table 1). As we aimed to establish how the term “virtual reality” has been defined and to which systems it has referred, we did not search papers mentioning other types of computerized rehabilitation (e.g., augmented reality, serious games).

Table 1.

Search strings used in different databases.

DATABASE SEARCH SYNTAX N
PubMed ((stroke*[Title/Abstract] OR cva*[Title/Abstract] OR poststroke[Title/Abstract] OR post-stroke[Title/Abstract] OR apoplex*[Title/Abstract]) OR ((brain[Title/Abstract] OR cerebell*[Title/Abstract] OR intracran*[Title/Abstract] OR intracerebral[Title/Abstract] OR vertebrobasilar[Title/Abstract]) AND (haemorrhag*[Title/Abstract] OR hemorrhag*[Title/Abstract] OR ischemi*[Title/Abstract] OR ischaemi*[Title/Abstract] OR infarct*[Title/Abstract] OR haematoma*[Title/Abstract] OR hematoma*[Title/Abstract] OR bleed*[Title/Abstract])) AND (rehabilit*[Title/Abstract] OR neurorehabilit*[Title/Abstract] OR treat*[Title/Abstract] OR heal*[Title/Abstract] OR restor*[Title/Abstract] OR cur*[Title/Abstract] OR improve*[Title/Abstract] OR recov*[Title/Abstract]) AND (virtual reality[Title/Abstract] OR Oculus Rift[All Fields] OR HTC Vive[All Fields] OR immersive[Title/Abstract] OR virtual environment[Title/Abstract]) AND (Review[Filter]) 100
Web of Science TS = ((stroke OR cva* OR poststroke OR post-stroke OR apoplex*) OR ((brain OR cerebell* OR intracran* OR intracerebral OR vertebrobasilar) AND (haemorrhag* OR hemorrhag* OR ischemi* OR ischaemi* OR infarct* OR haematoma* OR hematoma* OR bleed* OR damage))) #1 302
TS = (rehabilit* OR neurorehabilit* OR treat* OR heal* OR restor* OR cur* OR improve* OR recov*) #2
TS = (virtual reality OR Oculus Rift OR HTC Vive OR immersive OR virtual environment) #3
TS = (systematic AND review OR review OR meta-analysis OR literature AND search) #4
(#4 AND #3 AND #2 AND #1) OR ((#3 AND #2 AND #1) AND DOCUMENT TYPES: (Review))
Scopus TITLE-ABS-KEY (virtual AND reality) AND TITLE-ABS-KEY (rehabilit*) AND TITLE-ABS-KEY (stroke) AND (LIMIT-TO (DOCTYPE, “re”)) 121
Open in a new tab

Note: N = number of search results. We used the search string for “stroke” developed by Veerbeek et al. (2014). The search string for “rehabilitation” was iteratively optimized and the search string for “virtual reality” was restricted as we only wanted to include reviews that use the term “virtual reality”.

Reviews that discussed stroke patients or multiple patient groups including stroke patients were included. Reviews that mentioned multiple rehabilitation methods including VR treatments were included. Reviews that solely discussed other patient groups (e.g., Parkinson’s disease) or other rehabilitation methods without referring to them as “VR” (e.g., treadmill training) were excluded. Articles that presented previously unpublished data, editorials, study protocols and commentaries of single articles were excluded. Two raters (among which HH) independently evaluated the in- and exclusion criteria by subsequently screening the titles, the abstracts and the full texts (Figure 1). The two raters discussed disagreements (22% of all records) to reach a unanimous decision.

Figure 1.

Figure 1

Open in a new tab

Flow chart of in- and exclusion of reviews.

Data-extraction and data-analysis

Description of included reviews

We coded the main topic of the review, distinguishing reviews that either discussed efficacy, feasibility, design of VR therapies, or other topics (e.g., theoretical, methodological reviews and broad reviews without a specific focus). Moreover, we documented whether the review solely discussed VR rehabilitation versus multiple rehabilitation methods including VR. Additionally, we documented whether the review discussed a general patient group including stroke (i.e., general health, neurological disorders, acquired brain injury) or stroke patients only. We also documented which functional impairment the review discussed distinguishing upper or lower limb motor impairments, activities of daily living (ADL), memory, hemispatial neglect and/or pain perception. If reviews discussed multiple outcomes, we coded the functional impairments either as multiple domains (i.e., motor and cognition or motor and activities and participation), multiple motor impairments or multiple cognitive impairments.

To better understand the topics covered by reviews we performed a network and hierarchical cluster analysis. Each review characteristic was coded as present or absent per review. Then, we calculated the frequencies of each of these features and the frequency of co-occurrences of each pair of features using the R network package v1.15 (Butts et al., 2019; R Core Team, 2016). We used an undirected network with the weights based on the frequencies. The network was visualized with the ggnetwork package in R using the circular layout algorithm (Briatte, 2016). To categorize our reviews we used a hierarchical cluster analysis based on a binary distance measure from the stats R package (R Core Team, 2016) and visualized the results using the ggtree package in R (Yu et al., 2017).

How was VR defined?

First, we checked whether the term “virtual reality” was defined in the full texts of the reviews. Based on all reviews that defined VR, we developed a list of terms used in the definitions and then coded whether each term occurred in a definition. We used an undirected network analysis with weights based on the frequencies to find the terms used to define VR that co-occurred most often in reviews. We also documented whether authors distinguished different types of VR, how many subtypes were distinguished and how they were labelled.

Which technological systems, input- and output devices were considered VR?

Reviews sometimes referred to a single input- (i.e., the device that is used by the user to provide input to the system) or output device (i.e., the device that is used to provide input to the user by the system), or a technological system, which is a combination of input and output devices (e.g., IREX GestureTek). We documented the input and output devices and systems that were mentioned in the reviews. Descriptions of input and output devices can be found in Table S1 and Table S2. Only devices that were mentioned as examples of what the authors viewed as VR were included. If a review discussed assessment and rehabilitation, only rehabilitation systems were included. If a technological system was mentioned rather than specific devices, the devices were inferred from the system. For instance, if Nintendo Wii Sports was mentioned, we inferred that a regular 2D monitor and handheld motion controller were used. If the description of a system was not specific enough to be classified (e.g., “VR System”), we searched the cited articles for more information.

Results

A total of 366 unique records were identified, of which 121 met the inclusion criteria (Figure 1). To better understand the topics covered by reviews, we performed a network and hierarchical cluster analysis, which are techniques that are increasingly used to analyse and report results of systematic reviews (Ananiadou et al., 2009; Thomas et al., 2011).

Description of reviews

54% of reviews discussed stroke, 37% discussed neurological disorders, 5% discussed acquired brain injury and 3% discussed general health conditions (Figure 2A). 26% reviews discussed multiple functional domains, while 61% discussed motor impairments, either multiple motor impairments (23%) or upper- (17%) or lower limb impairments (22%) (Figure 3A). 7% of reviews discussed cognitive or perceptual impairments and 3% discussed activities of daily living (ADL). 61% reviews discussed efficacy/effectiveness, while only 12% discussed feasibility of therapies (Figure 2A). The majority of reviews (65%) discussed what they labelled as VR therapies, while the other reviews discussed multiple rehabilitation methods including VR (Figure 3A). The reviews were published between 2002 to 2019 (M = 2014, SD = 4) with 50% of reviews published since 2015 (Figure 2B).

Figure 2.

Figure 2

Open in a new tab

Review characteristics. Panel A represents the number of reviews for different review characteristics. Panel B depicts the number of reviews per publication year. GH = general health conditions, ND = neurological disorders, ABI = acquired brain injury, ADL = activities of daily living.

Figure 3.

Figure 3

Open in a new tab

Panel A represents the strongest and Panel B the weakest connections in the network of review characteristics. Colors represent the number of times that two features co-occurred across the included reviews. The width of the edges are scaled according to the number of reviews in which a pair of features co-occurred. The size of the nodes is scaled according to the number of reviews in which the feature occurred. GH = general health conditions, ND = neurological disorders, ABI = acquired brain injury, ADL = activities of daily living.

The most frequently connected review characteristics included efficacy, VR rehabilitation or multiple rehabilitation methods, multiple motor impairments or lower-limb motor impairments and stroke or neurological disorders in general (Figure 3).

The hierarchical cluster map in Figure 4 conveys similar information as Figure 3, but can guide readers to find reviews with certain features. Reviews placed closer to each other are more similar than reviews placed further apart. For instance, four studies forming a cluster discussed the efficacy of multiple rehabilitation methods for ADL in stroke patients. Also, among the bigger clusters, one cluster consists of reviews about the efficacy of VR rehabilitation for lower limb impairments in stroke patients or neurological disorders.

Figure 4.

Figure 4

Open in a new tab

Hierarchical clustering of reviews based on aetiology, functional impairment, the main question of the review and whether the review discussed VR or multiple rehabilitation methods including VR. A list of references for each article ID is available online (https://doi.org/10.6084/m9.figshare.11902614.v1). Labels in bold font with a * represent reviews that focused on VR rehabilitation, while labels in plain font represent reviews that focused on multiple rehabilitation methods including VR. ND = neurological disorders, ABI = acquired brain injury, ADL = activities of daily living.

How was VR defined?

64% of reviews defined “virtual reality”. In these reviews, the five most frequently used terms were an environment or world (92%), interaction (79%), simulation (63%), real-life or natural (56%) and computer-generated (54%) (Figure 5A). These terms co-occurred in more than 30% of these reviews (Figure 6A). 33% of reviews distinguished between different categories of VR. Non-immersive (65%) and immersive VR (60%) were the most common distinctions (Figure 5B).

Figure 5.

Figure 5

Open in a new tab

Terms used to define VR (A) and terms used to distinguish VR categories (B).

Figure 6.

Figure 6

Open in a new tab

Network of terms used in VR definitions. Panel A represents the strongest connections and panel B represents the weakest connections in the network. The color represents the number of times that two features co-occurred across the included reviews. Connections smaller than 10 reviews are not visualized. The width of the edges are scaled according to the number of reviews in which a pair of features co-occurred. The size of the nodes is scaled according to the number of reviews in which the feature occurred.

Which technological systems were considered VR?

85% of reviews mentioned technological systems or specific in- and output devices. We categorized the systems using Milgram’s taxonomy of mixed reality (Milgram et al., 1995; Milgram & Kishino, 1994). The first factor distinguished between VR, AR and AVR. The second dimension is the extent to which information is presented in an immersive way. Although immersion is best considered a continuum, we divided immersion in two categories. Semi-immersive systems (SIVR) use 2D displays with a limited field of view (i.e., window on the world displays) and the perspective on the environment does not change depending on head movements. Immersive systems (IVR) use head-mounted displays with head tracking or projections on large curved displays or rooms with multiple walls (i.e., CAVE) that almost provide a 360° field of view (Milgram & Kishino, 1994; Tieri et al., 2018).

We also categorized systems based on the main input device. In the SIVR category, we distinguished systems that use a motion-capture camera (SIVR video-capture), a handheld device which could either be a motion-capture or haptic device (SIVR handheld device), a motion-capture wearable device (SIVR wearable device), balance or force plates (SIVR force plates), robotic devices (SIVR robotic device), a bicycle (SIVR bicycle), treadmill (SIVR treadmill), steering wheels and pedals (SIVR drive simulator), and a computer mouse or keyboard (SIVR desktop systems) (Figure 7). In the IVR category, we further subdivided systems based on the input and output devices. Some IVR systems used a treadmill either coupled with an HMD or a CAVE (IVR treadmill). Other IVR systems used a force plate as input device and CAVE or HMD (IVR force plates). There were also systems that combined an HMD or CAVE with a steering wheel and pedals (IVR drive simulators) and there were systems using the newest generation of HMDs with wide field of view and head-tracking (IVR new generation HMD) or older generation HMDs with a limited field of view (IVR old generation HMD). A spreadsheet with examples of systems for each of these categories is available online (https://doi.org/10.6084/m9.figshare.11902614.v1).

Figure 7.

Figure 7

Open in a new tab

Taxonomy of mixed reality rehabilitation systems, based on the extent to which real and virtual information is mixed, the level of immersion and the main input device. AR = augmented reality, AVR = augmented virtual reality, IVR = immersive virtual reality, SIVR = semi-immersive virtual reality. HMD = head-mounted display. Permission to re-use images was provided by the original copyright owners in written format (Cho et al., 2015; Cordes et al., 2018; Gil-Gómez et al., 2011; Rizzo et al., 2004; Tobler-Ammann et al., 2017; van Kessel et al., 2013; Yin et al., 2016). The category SIVR desktop systems are not illustrated, because articles describing these systems typically do not include pictures of the hardware.

Which technological systems were most frequently labelled as VR?

The most frequently mentioned input devices were a motion-capture camera (67%) and motion-capture handheld device (49%). The two most popular output devices were a regular 2D monitor with a limited field of view (94%) and older generation HMDs (45%) (Figure 8). The most frequently mentioned mixed reality systems included SIVR systems using handheld devices (50%) and video-capture (50%), and AVR video-capture (46%). The next most popular systems were all SIVR in which patients provide input using a robotic device (41%), a balance/force plate (38%), or treadmill (31%). In the IVR category, systems using older generation HMDs with various interface devices (30%) or treadmill IVR systems (27%) were most frequently mentioned. The newest generation of HMDs were less frequently mentioned (7%).

Figure 8.

Figure 8

Open in a new tab

Frequency of different input and output devices (A) and mixed reality systems (B).

Reviews mentioned on average 4 different mixed reality systems (SD = 2, Range: 1–10), and an average of 2 mixed reality subtypes (i.e., AVR, IVR, SIVR, AR) (SD = 0.8, Range: 1–3). There were 4 reviews that only mentioned IVR, 25 that only mentioned SIVR, 1 that only mentioned AVR and 1 that only mentioned AR systems. All other reviews mentioned multiple types of mixed reality.

Differences in technological systems by functional impairment and publication year

We then assessed the number of times different mixed reality systems were discussed in relation to different functional impairments (Figure 9A) and publication year (Figure 9B). This analysis revealed that AR smart glasses were only discussed in the context of motor impairments, whereas AVR video-capture systems were discussed in relation to motor, cognitive impairments and ADL. SIVR and IVR systems using a treadmill, robotic device or force plate have mostly been discussed in the context of motor impairments, while IVR and SIVR drive simulators have mostly been discussed in relation to cognitive impairments or ADL. Finally, SIVR systems were the most frequently mentioned systems across publication years without a clear trend towards IVR systems (Figure 9B).

Figure 9.

Figure 9

Open in a new tab

Frequencies of different functional impairments for each mixed reality system (A) and frequencies of mixed reality subtype for each publication year (B).

Discussion

Our study revealed that VR was often not or only vaguely defined in review papers on VR post-stroke rehabilitation, confirming the lack of clarity in the use of VR terminology (Garrett et al., 2018; Perez-Marcos, 2018; Tieri et al., 2018). In these review papers, VR was typically described as a “computer-generated interactive virtual world that simulates the real world”, a definition that applies to many rehabilitation systems. Indeed, reviews often described multiple types of mixed reality systems. Moreover, we found large differences between systems labelled as “VR” in the level of immersion, the extent to which real-world and virtual information were mixed and the type of input devices used. These broad VR definitions may not be ideal as different systems may offer different opportunities, but likely face different challenges for post-stroke rehabilitation (Garrett et al., 2018; Iosa et al., 2012; Perez-Marcos, 2018; Tieri et al., 2018). Moreover, our systematic review revealed that few reviews discuss virtual reality post-stroke cognitive rehabilitation compared to post-stroke motor rehabilitation.

Efficacy and feasibility of VR stroke rehabilitation depends on the specific system

First, the viewing medium could influence the feasibility and efficacy of the rehabilitation. For instance, cybersickness has often been considered a contra-indication to use HMDs (Melo et al., 2018). Although studies using recent HMDs in older adults and stroke patients reported minimal cybersickness (Appel et al., 2020; Huygelier et al., 2019, 2020; Lee et al., 2020; Spreij et al., 2020), it is also known that cybersickness depends on several features of the IVR application (Davis et al., 2015; Porcino et al., 2017; Stanney & Hash, 1998; Weech et al., 2019) and end-users (Arns & Cerney, 2005).

Additionally, the viewing medium can affect multiple motor and cognitive aspects that could be relevant for rehabilitation. For instance, viewing a 3D environment with a HMD (40° horizontal field of view) reduced upper limb movement precision of healthy controls and stroke patients compared to when viewing the environment on a screen with polarizing glasses (Subramanian & Levin, 2011). Another study found that viewing a 2.5D environment (i.e., flat objects stacked behind each other) using shutter glasses improved reaching in-depth in neurological patients (van den Hoogen et al., 2012). Moreover, stroke patients and healthy controls experienced more body-ownership and presence when viewing a virtual body in a first-person perspective using a HMD than in a third-person perspective on a regular screen (Borrego et al., 2019). Another study found that navigation in a 3D environment was better when the environment was viewed with stereo-glasses versus on a 2D screen (Slobounov et al., 2015). The viewing medium may also matter for cognitive assessment and rehabilitation. For instance, one study showed that older adults performed worse in a virtual shopping memory task when assessed with an HMD than with a regular 2D computer monitor, while such effect of the viewing medium was not found for younger adults (Plechatá et al., 2019).

The design of the VR environment and visual feedback can be important too. Laver et al. (2012) stated that VR therapies to improve arm function, walking speed or independence in daily life specifically designed for the end-user group had better therapeutic effects than commercially available videogames. Moreover, the methods that are currently used to visualize patients’ movements vary widely across different systems (dos Santos et al., 2016). Some studies use AVR systems in which the patients view their own body, while other studies use SIVR systems in which a virtual avatar represents patients’ movements. However, few studies have directly compared the effects of these visualization methods on the feasibility and efficacy of therapy (dos Santos et al., 2016).

Finally, the device used to interact with the system may also influence efficacy and feasibility (Augstein & Neumayr, 2019; Milgram & Kishino, 1994; Tieri et al., 2018). Indeed, human-computer interaction studies have shown differences in usability of different input devices (Armbrüster et al., 2007; Bobeth et al., 2014; Chaparro et al., 1999; Gerling et al., 2013). Many rehabilitation systems used a motion-capture camera or wearable device as interaction device. These devices may be easier to use than handheld controllers. Indeed, some researchers stated that post-stroke rehabilitation games should be designed so that they can be played without patients using their hands (Alankus et al., 2010). However, this may not generalize to all handheld devices as one study found that stroke patients and healthy controls reported that it was easier to navigate a virtual maze using a handheld device that provided haptic feedback than when using a motion-tracking camera (Ramírez-Fernández et al., 2015). In sum, the specific devices and design of the mixed reality system are important to consider when evaluating the efficacy and feasibility of post-stroke rehabilitation systems. To facilitate conceptual clarity in post-stroke rehabilitation research, we propose new terminology to describe mixed reality systems.

A taxonomy of mixed reality rehabilitation systems

In older definitions, immersion or presence were key features of VR (Milgram & Kishino, 1994; Steuer, 1992). Recently, Tieri et al. (2018) advocated to restrict the definition of VR to refer to the most immersive systems that use a HMD or CAVE. However, since it has been so common to use the term “VR” to refer to SIVR systems, it may be more feasible to develop more specific terms to describe different systems. Indeed, others have suggested three categories: non-immersive (i.e., desktop monitor), semi-immersive (i.e., large screen monitor or projection with more than 60° field of view), and fully immersive VR (i.e., 360° display) (dos Santos et al., 2016; Kalawsky, 1996). However, given the importance of the input device and the large differences regarding input devices between systems labelled as “VR”, we suggest a more refined classification by extending the mixed reality continuum of Milgram (Milgram et al., 1995; Milgram & Kishino, 1994). This new classification of VR rehabilitation systems can provide a basis to standardize how researchers label their VR systems. In addition, given the broad definitions of VR in our field, it is important that the specific subtype of VR is more clearly labelled in the title and abstract of a paper. Note that, as we only included reviews that specifically mentioned the term “virtual reality”, our taxonomy only reflects the diversity in systems that were labelled as “virtual reality” and does not reflect all mixed reality systems used in post-stroke rehabilitation. Thus, our taxonomy is not necessarily exhaustive and new categories can be added over time. Moreover, there may still be considerable variability between systems within a single category. However, it can be an important step towards more conceptual clarity regarding VR technology in stroke rehabilitation.

The need to unify VR terminology within and across research domains

Although the current review focused on VR terminology within post-stroke rehabilitation, the findings have wider implications. VR has indeed received great interest in other domains, such as VR exposure therapy for phobias (Botella et al., 2017; Oing & Prescott, 2018), “virtual anaesthesia” (Wiederhold & Wiederhold, 2007), as biofeedback training for anxiety (van Rooij et al., 2016), as a tool to train social cognitive skills in children with autism spectrum disorders (Didehbani et al., 2016) or in combination with exercise equipment to alleviate depression (Zeng et al., 2018). Since that it is likely that the efficacy of such VR interventions is mediated by the sense of presence and realism offered, it is important to clearly describe VR systems as immersive or semi-immersive within each of these domains.

Indeed, recent reviews on VR anaesthesia and VR treatments for depression clearly specified which types of VR systems were included in their review, limiting it to systems using immersive 3D displays (e.g., Chan et al., 2018; Zeng et al., 2018). However, a clear terminology has not been widely implemented (Kardong-Edgren et al., 2019). For instance, Botella et al. (2017) reviewed the efficacy of VR exposure therapy for specific phobias, but did not specify which technological systems were considered “VR”. In addition, the reviews by Zeng et al. (2018) and Chan et al. (2018) only specified the type of VR in the methods sections of their papers, rather than in the abstract or title. Moreover, there is no consensus on how to label different VR systems, neither within nor across research domains (Kardong-Edgren et al., 2019; Tieri et al., 2018). Our mixed reality taxonomy can inspire consistent terminology to describe different mixed reality systems within and across research domains and help to unify VR terminology (Kardong-Edgren et al., 2019).

Conclusions

Our work demonstrates that VR is often not or only broadly defined, encompassing many technologies that differ regarding human-computer interaction modalities. This vague conceptualization made it unclear which mixed reality systems had been discussed in reviews. Our analysis revealed that many reviews on post-stroke rehabilitation discussed AVR video capture and SIVR, but very few discussed IVR systems. Moreover, few reviews focused on a specific type of mixed reality. Finally, our review informed a new data-driven taxonomy of mixed reality systems, which is expected to facilitate the communication amongst researchers and clinicians working with virtual reality.

Data Accessibility Statement

The dataset supporting the conclusions of this article is available on figshare, https://doi.org/10.6084/m9.figshare.11902548.v1, https://figshare.com/articles/Dataset/11902548.

The scripts used to analyze the data are available on figshare, https://doi.org/10.6084/m9.figshare.11902554.v1, https://figshare.com/articles/Data-analyses_scripts/11902554.

Additional Files

The additional files for this article can be found as follows:

Table S1.

Description and example(s) of input devices.

pb-61-1-1033-s1.pdf (160.7KB, pdf)
DOI: 10.5334/pb.1033.s1
Table S2.

Description of output devices.

pb-61-1-1033-s2.pdf (85.6KB, pdf)
DOI: 10.5334/pb.1033.s2

Acknowledgements

We would like to acknowledge Ioannis Doumas for his feedback on our manuscript.

Funding Statement

This work was funded by a research grant of the Flemish Fund for Scientific Research (FWO) awarded to H.H. (1171717N), R.v.E. and V.V.A. (G078915N) and an Odysseuss grant of the FWO to C.R.G. (G0H7718N).

Funding Information

This work was funded by a research grant of the Flemish Fund for Scientific Research (FWO) awarded to H.H. (1171717N), R.v.E. and V.V.A. (G078915N) and an Odysseuss grant of the FWO to C.R.G. (G0H7718N).

Competing interests

The author has no competing interests to declare.

Author Contributions

H.H. designed the study, collected and extracted data, analyzed data and wrote the manuscript. E.M. was involved in the initial phase of the study design. V.V., R.v.E. and CRG gave feedback on the study design and writing. All authors read and approved the final manuscript.

References

  • 1.Alankus, G., Lazar, A., May, M., & Kelleher, C. (2010). Towards customizable games for stroke rehabilitation. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2113–2122. DOI: 10.1145/1753326.1753649 [DOI] [Google Scholar]
  • 2.Ananiadou, S., Rea, B., Okazaki, N., Procter, R., & Thomas, J. (2009). Supporting Systematic Reviews Using Text Mining. Social Science Computer Review, 27(4), 509–523. DOI: 10.1177/0894439309332293 [DOI] [Google Scholar]
  • 3.Appel, L., Appel, E., Bogler, O., Wiseman, M., Cohen, L., Ein, N., Abrams, H. B., & Campos, J. L. (2020). Older Adults With Cognitive and/or Physical Impairments Can Benefit From Immersive Virtual Reality Experiences: A Feasibility Study. Frontiers in Medicine, 6. DOI: 10.3389/fmed.2019.00329 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Aravind, G., & Lamontagne, A. (2014). Perceptual and locomotor factors affect obstacle avoidance in persons with visuospatial neglect. Journal of NeuroEngineering and Rehabilitation, 11(38), 1–10. DOI: 10.1186/1743-0003-11-38 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Armbrüster, C., Sutter, C., & Ziefle, M. (2007). Notebook input devices put to the age test: The usability of trackpoint and touchpad for middle-aged adults. Ergonomics, 50(3), 426–445. DOI: 10.1080/00140130601127885 [DOI] [PubMed] [Google Scholar]
  • 6.Arns, L. L., & Cerney, M. M. (2005). The relationship between age and incidence of cybersickness among immersive environment users. IEEE Proceedings. VR 2005. Virtual Reality, 2005., 267–268. DOI: 10.1109/VR.2005.1492788 [DOI] [Google Scholar]
  • 7.Augstein, M., & Neumayr, T. (2019). A Human-Centered Taxonomy of Interaction Modalities and Devices. Interacting with Computers, 31(1), 27–58. DOI: 10.1093/iwc/iwz003 [DOI] [Google Scholar]
  • 8.Bernhardt, J., Chan, J., Nicola, I., & Collier, J. M. (2007). Little therapy, little physical activity: Rehabilitation within the first 14 days of organized stroke unit care. Journal of Rehabilitation Medicine, 39, 43–48. DOI: 10.2340/16501977-0013 [DOI] [PubMed] [Google Scholar]
  • 9.Bobeth, J., Schrammel, J., Deutsch, S., Klein, M., Drobics, M., Hochleitner, C., & Tscheligi, M. (2014). Tablet, gestures, remote control?: Influence of age on performance and user experience with iTV applications. Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, 139–146. DOI: 10.1145/2602299.2602315 [DOI] [Google Scholar]
  • 10.Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nature Reviews Neuroscience, 12(12), 752–762. DOI: 10.1038/nrn3122 [DOI] [PubMed] [Google Scholar]
  • 11.Borrego, A., Latorre, J., Alcañiz, M., & Llorens, R. (2019). Embodiment and Presence in Virtual Reality After Stroke. A Comparative Study With Healthy Subjects. Frontiers in Neurology, 10(1061), 1–8. DOI: 10.3389/fneur.2019.01061 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Botella, C., Fernández-Álvarez, J., Guillén, V., García-Palacios, A., & Baños, R. (2017). Recent Progress in Virtual Reality Exposure Therapy for Phobias: A Systematic Review. Current Psychiatry Reports, 19(7), 42. DOI: 10.1007/s11920-017-0788-4 [DOI] [PubMed] [Google Scholar]
  • 13.Briatte, F. (2016). ggnetwork: Geometries to plot Networks with “ggplot2” (0.5.1) [Computer software]. [Google Scholar]
  • 14.Butts, C. T., Hunter, D., Handcock, M., Bender-deMoll, S., Horner, J., & Wang, L. (2019). Package “network” (1.15) [Computer software]. [Google Scholar]
  • 15.Chan, E., Foster, S., Sambell, R., & Leong, P. (2018). Clinical efficacy of virtual reality for acute procedural pain management: A systematic review and meta-analysis. PLOS ONE, 13(7), e0200987. DOI: 10.1371/journal.pone.0200987 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Chaparro, A., Bohan, M., Fernandez, J., Choi, S. D., & Kattel, B. (1999). The impact of age on computer input device use: Psychophysical and physiological measures. International Journal of Industrial Ergonomics, 24(5), 503–513. DOI: 10.1016/S0169-8141(98)00077-8 [DOI] [Google Scholar]
  • 17.Cho, K. H., Kim, M. K., Lee, H.-J., & Lee, W. H. (2015). Virtual Reality Training with Cognitive Load Improves Walking Function in Chronic Stroke Patients. The Tohoku Journal of Experimental Medicine, 236(4), 273–280. DOI: 10.1620/tjem.236.273 [DOI] [PubMed] [Google Scholar]
  • 18.Christensen, D., Johnsen, S. P., Watt, T., Harder, I., Kirkevold, M., & Andersen, G. (2008). Dimensions of Post-Stroke Fatigue: A Two-Year Follow-Up Study. Cerebrovascular Diseases, 26(2), 134–141. DOI: 10.1159/000139660 [DOI] [PubMed] [Google Scholar]
  • 19.Cordes, C., Heutink, J., Brookhuis, K. A., Brouwer, W. H., & Melis-Dankers, B. J. M. (2018). Driving slow motorised vehicles with visual impairment—A simulator study. Cogent Psychology, 5(1), 1485473. DOI: 10.1080/23311908.2018.1485473 [DOI] [Google Scholar]
  • 20.Davis, S., Nesbitt, K., & Nalivaiko, E. (2015). Comparing the onset of cybersickness using the Oculus Rift and two virtual roller coasters. Proceedings of the 11th Australasian Conference on Interactive Entertainment (IE 2015), 167, 1–12. [Google Scholar]
  • 21.Didehbani, N., Allen, T., Kandalaft, M., Krawczyk, D., & Chapman, S. (2016). Virtual Reality Social Cognition Training for children with high functioning autism. Computers in Human Behavior, 62, 703–711. DOI: 10.1016/j.chb.2016.04.033 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.dos Santos, L. F., Christ, O., Mate, K., Schmidt, H., Krueger, J., & Dohle, C. (2016). Movement visualisation in virtual reality rehabilitation of the lower limb: A systematic review. Biomedical Engineering Online, 15, 144. DOI: 10.1186/s12938-016-0289-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Douiri, A., Rudd, G. A., & Wolfe, C. D. A. (2013). Prevalence of Poststroke Cognitive Impairment. Stroke, 44(1), 138–145. DOI: 10.1161/STROKEAHA.112.670844 [DOI] [PubMed] [Google Scholar]
  • 24.Dvorkin, A. Y., Bogey, R. A., Harvey, R. L., & Patton, J. L. (2012). Mapping the Neglected Space Gradients of Detection Revealed by Virtual Reality. Neurorehabilitation and Neural Repair, 26(2), 120–131. DOI: 10.1177/1545968311410068 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Edmans, J. A., & Lincoln, N. B. (1987). The frequency of perceptual deficits after stroke. Clinical Rehabilitation, 1(4), 273–281. DOI: 10.1177/030802268905200706 [DOI] [Google Scholar]
  • 26.Espárrago, L. G., Castilla-Guerra, L., Moreno, F. M. C., Doblado, R. S., & Hernández, J. M. D. (2015). Post-stroke depression: An update. Neurología (English Edition), 30(1), 23–31. DOI: 10.1016/j.nrleng.2012.06.006 [DOI] [PubMed] [Google Scholar]
  • 27.Garrett, B., Taverner, T., Gromala, D., Tao, G., Cordingley, E., & Sun, C. (2018). Virtual Reality Clinical Research: Promises and Challenges. Jmir Serious Games, 6(4), e10839. DOI: 10.2196/10839 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Gerling, K. M., Dergousoff, K. K., & Mandryk, R. L. (2013). Is Movement Better?: Comparing Sedentary and Motion-based Game Controls for Older Adults. Proceedings of Graphics Interface 2013, 133–140. http://dl.acm.org/citation.cfm?id=2532129.2532153 [Google Scholar]
  • 29.Gil-Gómez, J.-A., Lloréns, R., Alcañiz, M., & Colomer, C. (2011). Effectiveness of a Wii balance board-based system (eBaViR) for balance rehabilitation: A pilot randomized clinical trial in patients with acquired brain injury. Journal of Neuroengineering and Rehabilitation, 8, 30. DOI: 10.1186/1743-0003-8-30 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Hesse, S., & Werner, C. (2003). Poststroke Motor Dysfunction and Spasticity. CNS Drugs, 17(15), 1093–1107. DOI: 10.2165/00023210-200317150-00004 [DOI] [PubMed] [Google Scholar]
  • 31.Hochstenbach, J., Prigatano, G., & Mulder, T. (2005). Patients’ and Relatives’ Reports of Disturbances 9 Months After Stroke: Subjective Changes in Physical Functioning, Cognition, Emotion, and Behavior. Archives of Physical Medicine and Rehabilitation, 86(8), 1587–1593. DOI: 10.1016/j.apmr.2004.11.050 [DOI] [PubMed] [Google Scholar]
  • 32.Huygelier, H., Schraepen, B., Lafosse, C., Vaes, N., Schillebeeckx, F., Michiels, K., Note, E., Vanden Abeele, V., van Ee, R., & Gillebert, C. R. (2020). An immersive virtual reality game to train spatial attention orientation after stroke: A feasibility study. Applied Neuropsychology: Adult, 1–21. DOI: 10.1080/23279095.2020.1821030 [DOI] [PubMed] [Google Scholar]
  • 33.Huygelier, H., Schraepen, B., van Ee, R., Vanden Abeele, V., & Gillebert, C. R. (2019). Acceptance of immersive head-mounted virtual reality in older adults. Scientific Reports, 9(1), 4519. DOI: 10.1038/s41598-019-41200-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Institute for Health Metrics and Evaluation (IHME). (2018). Findings from the Global Burden of Disease Study 2017. IHME, 2018. http://www.healthdata.org/policy-report/findings-global-burden-disease-study-2017 [Google Scholar]
  • 35.Iosa, M., Morone, G., Fusco, A., Bragoni, M., Coiro, P., Multari, M., Venturiero, V., De Angelis, D., Pratesi, L., & Paolucci, S. (2012). Seven Capital Devices for the Future of Stroke Rehabilitation. Stroke Research and Treatment, 2012. DOI: 10.1155/2012/187965 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Jaillard, A., Naegele, B., Trabucco-Miguel, S., LeBas, J. F., & Hommel, M. (2009). Hidden Dysfunctioning in Subacute Stroke. Stroke, 40(7), 2473–2479. DOI: 10.1161/STROKEAHA.108.541144 [DOI] [PubMed] [Google Scholar]
  • 37.Kalawsky, R. S. (1996). Exploiting Virtual Reality techniques in Education and Training: Technological Issues. https://ci.nii.ac.jp/naid/10010070775/
  • 38.Kao, P.-C., Dingwell, J. B., Higginson, J. S., & Binder-Macleod, S. (2014). Dynamic instability during post-stroke hemiparetic walking. Gait & Posture, 40(3), 457–463. DOI: 10.1016/j.gaitpost.2014.05.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Kardong-Edgren, S. (Suzie), Farra, S. L., Alinier, G., & Young, H. M. (2019). A Call to Unify Definitions of Virtual Reality. Clinical Simulation in Nursing, 31, 28–34. DOI: 10.1016/j.ecns.2019.02.006 [DOI] [Google Scholar]
  • 40.Katie, D., Wolfe, C. D. A., Markus, A. B., & Christopher, M. (2009). What Are the Social Consequences of Stroke for Working-Aged Adults? Stroke, 40(6), e431–e440. DOI: 10.1161/STROKEAHA.108.534487 [DOI] [PubMed] [Google Scholar]
  • 41.Kauhanen, M.-L., Korpelainen, J. T., Hiltunen, P., Määttä, R., Mononen, H., Brusin, E., Sotaniemi, K. A., & Myllylä, V. V. (2000). Aphasia, Depression, and Non-Verbal Cognitive Impairment in Ischaemic Stroke. Cerebrovascular Diseases, 10(6), 455–461. DOI: 10.1159/000016107 [DOI] [PubMed] [Google Scholar]
  • 42.Lang, C. E., MacDonald, J. R., Reisman, D. S., Boyd, L., Kimberley, T. J., Schindler-Ivens, S. M., Hornby, T. G., Ross, S. A., & Scheets, P. L. (2009). Observation of Amounts of Movement Practice Provided During Stroke Rehabilitation. Archives of Physical Medicine and Rehabilitation, 90(10), 1692–1698. DOI: 10.1016/j.apmr.2009.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Langhorne, P., Coupar, F., & Pollock, A. (2009). Motor recovery after stroke: A systematic review. The Lancet Neurology, 8(8), 741–754. DOI: 10.1016/S1474-4422(09)70150-4 [DOI] [PubMed] [Google Scholar]
  • 44.Laver, K., George, S., Thomas, S., Deutsch, J. E., & Crotty, M. (2012). Cochrane review: Virtual reality for stroke rehabilitation. European Journal of Physical and Rehabilitation Medicine, 48(3), 523–530. DOI: 10.1002/14651858.CD008349.pub2 [DOI] [PubMed] [Google Scholar]
  • 45.Lee, S. H., Jung, H.-Y., Yun, S. J., Oh, B.-M., & Seo, H. G. (2020). Upper Extremity Rehabilitation Using Fully Immersive Virtual Reality Games With a Head Mount Display: A Feasibility Study. PM&R, 12(3), 257–262. DOI: 10.1002/pmrj.12206 [DOI] [PubMed] [Google Scholar]
  • 46.Lohse, K. R., Lang, C. E., & Boyd, L. A. (2014). Is More Better? Using Metadata to Explore Dose–Response Relationships in Stroke Rehabilitation. Stroke, 45(7), 2053–2058. DOI: 10.1161/STROKEAHA.114.004695 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Makin, S. D. J., Turpin, S., Dennis, M. S., & Wardlaw, J. M. (2013). Cognitive impairment after lacunar stroke: Systematic review and meta-analysis of incidence, prevalence and comparison with other stroke subtypes. Journal of Neurology, Neurosurgery & Psychiatry, 84(8), 893–900. DOI: 10.1136/jnnp-2012-303645 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Massetti, T., da Silva, T. D., Crocetta, T. B., Guarnieri, R., de Freitas, B. L., Bianchi Lopes, P., Watson, S., Tonks, J., & de Mello Monteiro, C. B. (2018). The Clinical Utility of Virtual Reality in Neurorehabilitation: A Systematic Review. Journal of Central Nervous System Disease, 10, 1179573518813541. DOI: 10.1177/1179573518813541 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Mayo, N. E., Fellows, L. K., Scott, S. C., Cameron, J., & Wood-Dauphinee, S. (2009). A Longitudinal View of Apathy and Its Impact After Stroke. Stroke, 40(10), 3299–3307. DOI: 10.1161/STROKEAHA.109.554410 [DOI] [PubMed] [Google Scholar]
  • 50.McDowd, J. M., Filion, D. L., Pohl, P. S., Richards, L. G., & Stiers, W. (2003). Attentional Abilities and Functional Outcomes Following Stroke. The Journals of Gerontology: Series B, 58(1), P45–P53. DOI: 10.1093/geronb/58.1.P45 [DOI] [PubMed] [Google Scholar]
  • 51.Melo, M., Vasconcelos-Raposo, J., & Bessa, M. (2018). Presence and cybersickness in immersive content: Effects of content type, exposure time and gender. Computers & Graphics, 71, 159–165. DOI: 10.1016/j.cag.2017.11.007 [DOI] [Google Scholar]
  • 52.Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12), 1321–1329. [Google Scholar]
  • 53.Milgram, P., Takemura, H., Utsumi, A., & Kishino, F. (1995). Augmented reality: A class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies, 2351, 282–293. DOI: 10.1117/12.197321 [DOI] [Google Scholar]
  • 54.Miller, K. K., Porter, R. E., DeBaun-Sprague, E., Puymbroeck, M. V., & Schmid, A. A. (2017). Exercise after Stroke: Patient Adherence and Beliefs after Discharge from Rehabilitation. Topics in Stroke Rehabilitation, 24(2), 142–148. DOI: 10.1080/10749357.2016.1200292 [DOI] [PubMed] [Google Scholar]
  • 55.Mirelman, A., Bonato, P., & Deutsch, J. E. (2009). Effects of Training With a Robot-Virtual Reality System Compared With a Robot Alone on the Gait of Individuals After Stroke. Stroke, 40(1), 169–174. DOI: 10.1161/STROKEAHA.108.516328 [DOI] [PubMed] [Google Scholar]
  • 56.Myers, R. L., & Bierig, T. A. (2000). Virtual Reality and Left Hemineglect: A Technology for Assessment and Therapy. CyberPsychology & Behavior, 3(3), 465–468. DOI: 10.1089/10949310050078922 [DOI] [Google Scholar]
  • 57.Nelson, L. D., Cicchetti, D., Satz, P., Stern, S., Sowa, M., Cohen, S., Mitrushina, M., & van Gorp, W. (1993). Emotional sequelae of stroke. Neuropsychology, 7(4), 553–560. DOI: 10.1037/0894-4105.7.4.553 [DOI] [PubMed] [Google Scholar]
  • 58.Nys, G. M. S., Zandvoort, M. J. E. van, Kort, P. L. M. de, Jansen, B. P. W., Haan, E. H. F. de, & Kappelle, L. J. (2007). Cognitive Disorders in Acute Stroke: Prevalence and Clinical Determinants. Cerebrovascular Diseases, 23(5–6), 408–416. DOI: 10.1159/000101464 [DOI] [PubMed] [Google Scholar]
  • 59.Oing, T., & Prescott, J. (2018). Implementations of Virtual Reality for Anxiety-Related Disorders: Systematic Review. JMIR Serious Games, 6(4), e10965. DOI: 10.2196/10965 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Ong, Y.-H., Brown, M. M., Robinson, P., Plant, G. T., Husain, M., & Leff, A. P. (2012). Read-Right: A “web app” that improves reading speeds in patients with hemianopia. Journal of Neurology, 259(12), 2611–2615. DOI: 10.1007/s00415-012-6549-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Otterman, N. M., van der Wees, P. J., Bernhardt, J., & Kwakkel, G. (2012). Physical Therapists’ Guideline Adherence on Early Mobilization and Intensity of Practice at Dutch Acute Stroke Units. Stroke, 43(9), 2395–2401. DOI: 10.1161/STROKEAHA.112.660092 [DOI] [PubMed] [Google Scholar]
  • 62.Parker, A. M., Lord, R. K., & Needham, D. M. (2013). Increasing the dose of acute rehabilitation: Is there a benefit? BMC Medicine, 11(1), 199. DOI: 10.1186/1741-7015-11-199 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Pearce, S. C., Stolwyk, R. J., New, P. W., & Anderson, C. (2016). Sleep disturbance and deficits of sustained attention following stroke. Journal of Clinical and Experimental Neuropsychology, 38(1), 1–11. DOI: 10.1080/13803395.2015.1078295 [DOI] [PubMed] [Google Scholar]
  • 64.Pedroli, E., Serino, S., Cipresso, P., Pallavicini, F., & Riva, G. (2015). Assessment and rehabilitation of neglect using virtual reality: A systematic review. Frontiers in Behavioral Neuroscience, 9, 226. DOI: 10.3389/fnbeh.2015.00226 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Perez-Marcos, D. (2018). Virtual reality experiences, embodiment, videogames and their dimensions in neurorehabilitation. Journal of Neuroengineering and Rehabilitation, 15, 113. DOI: 10.1186/s12984-018-0461-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Plechatá, A., Sahula, V., Fayette, D., & Fajnerová, I. (2019). Age-Related Differences With Immersive and Non-immersive Virtual Reality in Memory Assessment. Frontiers in Psychology, 10. DOI: 10.3389/fpsyg.2019.01330 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Porcino, T. M., Clua, E., Trevisan, D., Vasconcelos, C. N., & Valente, L. (2017). Minimizing cyber sickness in head mounted display systems: Design guidelines and applications. 2017 IEEE 5th International Conference on Serious Games and Applications for Health (SeGAH), 1–6. DOI: 10.1109/SeGAH.2017.7939283 [DOI] [Google Scholar]
  • 68.Porras, D. C., Siemonsma, P., Inzelberg, R., Zeilig, G., & Plotnik, M. (2018). Advantages of virtual reality in the rehabilitation of balance and gait Systematic review. Neurology, 90(22), 1017–1025. DOI: 10.1212/WNL.0000000000005603 [DOI] [PubMed] [Google Scholar]
  • 69.R Core Team. (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing. [Google Scholar]
  • 70.Raghavan, P. (2015). Upper Limb Motor Impairment After Stroke. Physical Medicine and Rehabilitation Clinics, 26(4), 599–610. DOI: 10.1016/j.pmr.2015.06.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Ramírez-Fernández, C., Morán, A. L., & García-Canseco, E. (2015). Haptic Feedback in Motor Hand Virtual Therapy Increases Precision and Generates Less Mental Workload. Proceedings of the 9th International Conference on Pervasive Computing Technologies for Healthcare. Istanbul, Turkey. DOI: 10.4108/icst.pervasivehealth.2015.260242 [DOI] [Google Scholar]
  • 72.Rizzo, A., & Kim, G. J. (2005). A SWOT Analysis of the Field of Virtual Reality Rehabilitation and Therapy. Presence: Teleoperators and Virtual Environments, 14(2), 119–146. DOI: 10.1162/1054746053967094 [DOI] [Google Scholar]
  • 73.Rizzo, A. A., Cohen, I., Weiss, P. L., Kim, J. G., Yeh, S. C., Zali, B., & Hwang, J. (2004). Design and development of virtual reality based perceptual-motor rehabilitation scenarios. Conference Proceedings: … Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, 7, 4852–4855. DOI: 10.1109/IEMBS.2004.1404342 [DOI] [PubMed] [Google Scholar]
  • 74.Rizzo, A. A., Schultheis, M., Kerns, K. A., & Mateer, C. (2004). Analysis of assets for virtual reality applications in neuropsychology. Neuropsychological Rehabilitation, 14(1–2), 207–239. DOI: 10.1080/09602010343000183 [DOI] [Google Scholar]
  • 75.Rose, F. D., Brooks, B. M., & Rizzo, A. A. (2005). Virtual reality in brain damage rehabilitation: Review. Cyberpsychology & Behavior, 8(3), 241–262. DOI: 10.1089/cpb.2005.8.241 [DOI] [PubMed] [Google Scholar]
  • 76.Sibley, K. M., Tang, A., Patterson, K. K., Brooks, D., & McIlroy, W. E. (2009). Changes in spatiotemporal gait variables over time during a test of functional capacity after stroke. Journal of NeuroEngineering and Rehabilitation, 6(1), 27. DOI: 10.1186/1743-0003-6-27 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Slater, M. (2003). A note on presence terminology. Presence Connect, 3(3), 1–5. [Google Scholar]
  • 78.Slobounov, S. M., Ray, W., Johnson, B., Slobounov, E., & Newell, K. M. (2015). Modulation of cortical activity in 2D versus 3D virtual reality environments: An EEG study. International Journal of Psychophysiology, 95(3), 254–260. DOI: 10.1016/j.ijpsycho.2014.11.003 [DOI] [PubMed] [Google Scholar]
  • 79.Snaphaan, L., Werf, S. van der, & Leeuw, F.-E. de. (2011). Time course and risk factors of post-stroke fatigue: A prospective cohort study. European Journal of Neurology, 18(4), 611–617. DOI: 10.1111/j.1468-1331.2010.03217.x [DOI] [PubMed] [Google Scholar]
  • 80.Spreij, L. A., Visser-Meily, J. M. A., Sibbel, J., Gosselt, I. K., & Nijboer, T. C. W. (2020). Feasibility and user-experience of virtual reality in neuropsychological assessment following stroke. Neuropsychological Rehabilitation, 1–21. DOI: 10.1080/09602011.2020.1831935 [DOI] [PubMed] [Google Scholar]
  • 81.Stanney, K. M., & Hash, P. (1998). Locus of User-Initiated Control in Virtual Environments: Influences on Cybersickness. Presence: Teleoperators and Virtual Environments, 7(5), 447–459. DOI: 10.1162/105474698565848 [DOI] [Google Scholar]
  • 82.Steuer, J. (1992). Defining Virtual Reality: Dimensions Determining Telepresence. Journal of Communication, 42(4), 73–93. DOI: 10.1111/j.1460-2466.1992.tb00812.x [DOI] [Google Scholar]
  • 83.Subramanian, S. K., & Levin, M. F. (2011). Viewing medium affects arm motor performance in 3D virtual environments. Journal of NeuroEngineering and Rehabilitation, 8(1), 36. DOI: 10.1186/1743-0003-8-36 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 84.Sue-Min, L., Studenski, S., Duncan, P. W., & Subashan, P. (2002). Persisting Consequences of Stroke Measured by the Stroke Impact Scale. Stroke, 33(7), 1840–1844. DOI: 10.1161/01.STR.0000019289.15440.F2 [DOI] [PubMed] [Google Scholar]
  • 85.Thomas, J., McNaught, J., & Ananiadou, S. (2011). Applications of text mining within systematic reviews. Research Synthesis Methods, 2(1), 1–14. DOI: 10.1002/jrsm.27 [DOI] [PubMed] [Google Scholar]
  • 86.Tiedemann, A., Sherrington, C., Dean, C. M., Rissel, C., Lord, S. R., Kirkham, C., & O’Rourke, S. D. (2012). Predictors of Adherence to a Structured Exercise Program and Physical Activity Participation in Community Dwellers after Stroke. Stroke Research and Treatment, 2012, 1–8. DOI: 10.1155/2012/136525 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 87.Tieri, G., Morone, G., Paolucci, S., & Iosa, M. (2018). Virtual reality in cognitive and motor rehabilitation: Facts, fiction and fallacies. Expert Review of Medical Devices, 15(2), 107–117. DOI: 10.1080/17434440.2018.1425613 [DOI] [PubMed] [Google Scholar]
  • 88.Tobler-Ammann, B. C., Surer, E., Knols, R. H., Borghese, N. A., & Bruin, E. D. de. (2017). User Perspectives on Exergames Designed to Explore the Hemineglected Space for Stroke Patients With Visuospatial Neglect: Usability Study. JMIR Serious Games, 5(3), e18. DOI: 10.2196/games.8013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 89.van den Hoogen, W., Feys, P., Lamers, I., Coninx, K., Notelaers, S., Kerkhofs, L., & IJsselsteijn, W. (2012). Visualizing the third dimension in virtual training environments for neurologically impaired persons: Beneficial or disruptive? Journal of NeuroEngineering and Rehabilitation, 9(1), 73. DOI: 10.1186/1743-0003-9-73 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 90.van Kessel, M. E., Geurts, A. C. H., Brouwer, W. H., & Fasotti, L. (2013). Visual Scanning Training for Neglect after Stroke with and without a Computerized Lane Tracking Dual Task. Frontiers in Human Neuroscience, 7, 1–11. DOI: 10.3389/fnhum.2013.00358 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 91.van Rooij, M., Lobel, A., Harris, O., Smit, N., & Granic, I. (2016). DEEP: A Biofeedback Virtual Reality Game for Children At-risk for Anxiety. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 1989–1997. DOI: 10.1145/2851581.2892452 [DOI] [Google Scholar]
  • 92.Veerbeek, J. M., Van Wegen, E., Van Peppen, R., Van Der Wees, P. J., Hendriks, E., Rietberg, M., & Kwakkel, G. (2014). What is the evidence for physical therapy poststroke? A systematic review and meta-analysis. PLoS ONE, 9(2). Scopus. DOI: 10.1371/journal.pone.0087987 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 93.Verheyden, G., Nieuwboer, A., De Wit, L., Thijs, V., Dobbelaere, J., Devos, H., Severijns, D., Vanbeveren, S., & De Weerdt, W. (2008). Time Course of Trunk, Arm, Leg, and Functional Recovery After Ischemic Stroke. Neurorehabilitation and Neural Repair, 22(2), 173–179. DOI: 10.1177/1545968307305456 [DOI] [PubMed] [Google Scholar]
  • 94.Weber, L. M., Nilsen, D. M., Gillen, G., Yoon, J., & Stein, J. (2019). Immersive virtual reality mirror therapy for upper limb recovery following stroke: A pilot study. American Journal of Physical Medicine & Rehabilitation, 98(9), 783–788. DOI: 10.1097/PHM.0000000000001190 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 95.Weech, S., Kenny, S., & Barnett-Cowan, M. (2019). Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review. Frontiers in Psychology, 10. DOI: 10.3389/fpsyg.2019.00158 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 96.Wentink, M. M., Meesters, J., Berger, M. A. M., Kloet, A. J. de, Stevens, E., Band, G. P. H., Kromme, C. H., Wolterbeek, R., Goossens, P. H., & Vlieland, T. P. M. V. (2018). Adherence of stroke patients with an online brain training program: The role of health professionals’ support. Topics in Stroke Rehabilitation, 25(5), 359–365. DOI: 10.1080/10749357.2018.1459362 [DOI] [PubMed] [Google Scholar]
  • 97.Wiederhold, M. D., & Wiederhold, B. K. (2007). Virtual Reality and Interactive Simulation for Pain Distraction. Pain Medicine, 8(suppl_3), S182–S188. DOI: 10.1111/j.1526-4637.2007.00381.x [DOI] [Google Scholar]
  • 98.Yin, C., Hsueh, Y.-H., Yeh, C.-Y., Lo, H.-C., & Lan, Y.-T. (2016). A Virtual Reality-Cycling Training System for Lower Limb Balance Improvement. BioMed Research International, 2016, 1–10. DOI: 10.1155/2016/9276508 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 99.Yu, G., Smith, D., Zhu, H., Guan, Y., & Lam, T. (2017). ggtree: An R package for visualization and annotation of phylogenetic trees with their covariates and other associated data. 8, 28–36. DOI: 10.1093/molbev/msy194 [DOI] [Google Scholar]
  • 100.Zeng, N., Pope, Z., Lee, J. E., & Gao, Z. (2018). Virtual Reality Exercise for Anxiety and Depression: A Preliminary Review of Current Research in an Emerging Field. Journal of Clinical Medicine, 7(3), 42. DOI: 10.3390/jcm7030042 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Table S1.

Description and example(s) of input devices.

pb-61-1-1033-s1.pdf (160.7KB, pdf)
DOI: 10.5334/pb.1033.s1
Table S2.

Description of output devices.

pb-61-1-1033-s2.pdf (85.6KB, pdf)
DOI: 10.5334/pb.1033.s2

Data Availability Statement

The dataset supporting the conclusions of this article is available on figshare, https://doi.org/10.6084/m9.figshare.11902548.v1, https://figshare.com/articles/Dataset/11902548.

The scripts used to analyze the data are available on figshare, https://doi.org/10.6084/m9.figshare.11902554.v1, https://figshare.com/articles/Data-analyses_scripts/11902554.

Articles from Psychologica Belgica are provided here courtesy of Ubiquity Press

RESOURCES