A systematic review of the use of virtual reality and its effects on cognition in individuals with neurocognitive disorders

Abstract

Introduction

Virtual reality (VR) interventions are increasingly used in individuals with brain injuries. The objective of this study was to determine the effects of VR on overall cognitive functioning in individuals with neurocognitive disorders (NCDs).

Methods

Using Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, a systematic review of the published literature on immersive and nonimmersive VR technologies targeting cognition in minor and major NCDs was conducted: (PROSPERO registration number: CRD42019121953).

Results

A total of 22 studies were included in the review, for an aggregated sample of 564 individuals with NCDs. Most of the studies were conducted on patients who had stroke (27.3%), followed by mild cognitive impairment (22.7%) and Alzheimer's disease (13.6%). VR interventions used for cognitive rehabilitation suggested to improve cognition (e.g. memory, dual tasking, and visual attention), and secondarily to psychological functioning (e.g. reduction of anxiety, higher levels of well-being, and increased use of coping strategies).

Conclusion

VR interventions are useful to improve cognition and psychological symptoms in NCDs.

1. Introduction

Neurocognitive disorders (NCDs), defined in accordance with the Diagnostic and Statistical Manual, Edition 5: DSM-5 criteria, are an umbrella term for cognitive disorders formerly known as dementia [1,2]. Some examples include major NCDs due to Alzheimer's disease (AD), vascular disease, or Parkinson's disease. Mild neurocognitive impairment, also termed mild cognitive impairment, can be a prodromal state of major NCD. Both incidence and prevalence increase with age [3], with substantial costs representing a challenge for the economy [4]. For individuals with NCDs, changes in cognition, behavior, and emotions can lead to progressive and irreversible functional limitations, affecting everyday activities and autonomy [5].

Virtual reality (VR) is an emerging technology that digitally provides a three-dimensional environment, allowing persons to interact, provide sensory inputs, and track changes [6]. VR can be presented in fully immersive (high level of immersion) or nonimmersive environments (low level of immersion) [7,8]. Immersion provides a sense of presence in the virtual world with an immersive display device (e.g. head-mounted display) and an interactive device (e.g. joystick, glove). VR has been used in health care and education for both rehabilitation and training purposes [[9], [10], [11], [12]]. VR technologies are an innovative rehabilitation approach to minimize the negative impact of NCDs on individuals, families, and society.

VR has been successfully used in the elderly and in individuals who had stroke and Parkinson's disease to enhance the ability to perform activities of daily living [13], in individuals at high risk for cognitive decline [14] to reduce anxiety in older adults consulting for the first time in a memory clinic [15], and in individuals with NCDs for memory rehabilitation [16]. Interestingly, VR has been used in the diagnosis, cognitive training, and caregiver education for major NCD due to AD [17,18], to improve executive function in individuals with NCDs due to traumatic brain injury [19], cognitive rehabilitation of mild cognitive impairment [[20], [21], [22]], and stroke [23,24]. The evidence is sparse, and a comprehensive picture of the effects of VR interventions on NCDs is needed. That is, an overall description of the use of VR in the cognitive rehabilitation of NCDs remains predominantly undetermined. We aim to systematically review VR studies focused on cognitive rehabilitation in NCDs. As a secondary objective, we aimed to identify changes in psychological functioning (e.g. improved well-being and reduction of emotional problems) after VR interventions in NCDs.

2. Methods

2.1. Search strategy and information sources

A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) systematic review was conducted using the keywords (“Dementia” OR “Cognitive impairment”) and (“Virtual reality” OR “Virtual-based”) with different combinations of those terms in the databases MEDLINE/PubMed, Cochrane Central Register of Controlled Trials (CENTRAL), EMBASE, LILACS, SciELO, PEDro, CINAHL, and Scopus from inception to November 2018. A librarian and a clinical neuropsychologist helped the team to refine the search strategy and the search was cross-validated by a second librarian. A reference manager software was used – EndNote X9 [25].

The systematic review has been registered in PROSPERO: (registration number: CRD42019121953).

2.2. Study selection

Eligibility of the studies followed inclusion criteria: (1) VR interventions conducted in adults with NCDs of different etiologies (e.g. AD, vascular disease, Parkinson's disease, and mild cognitive impairment); (2) Studies conducted in the community, hospital, or residential care; (3) Studies measuring cognition before and after VR interventions. Interventions included the use of any type of immersive and nonimmersive VR technology targeting cognition in individuals with NCDs; (4) Quantitative, qualitative, and mixed studies; (5) Feasibility studies (i.e., user acceptance and adverse effects); and (6) Studies available in the English language. We excluded studies based on the following characteristics: (1) Studies conducted in individuals with traumatic brain injury and delirium; (2) Studies focusing on family or professional caregivers of individuals with NCD or outcome measures that focused on family members and professional caregivers of individuals living with NCD; (3) Research protocols and reviews; (4) Expert letters, opinion pieces, notes, editorials, and book chapters; and (5) Conference papers and abstracts.

2.3. Data extraction and synthesis

First, titles and abstracts were screened by two independent reviewers (i.e., a researcher in VR and a clinical neuropsychologist) based on the eligibility criteria. Studies meeting the inclusion criteria, or studies that were unclear, were retained for full-text review. Discrepancies were resolved by consensus and a third reviewer for determination. Second, two reviewers independently extracted the following information from each study: Publication data (i.e., author, year, and title), study design, objectives, setting, participants (e.g. sample size, mean age, sex, ethnicity, diagnosis, NCD severity, NCD duration, and the demographic information for control groups when available), VR intervention (i.e., name of the VR application, technical information, subjective and objective level of immersion based on published guidelines [7], the number of VR sessions and frequency, length of each VR session, and VR overall mean duration), outcome measures, results, user acceptance, adverse effects, generalization, and the general conclusion.

2.4. Levels of VR immersion

The level of immersive capability of VR technologies was assessed using five criteria [7,8]: (1) Inclusiveness that refers to whether a VR technology eliminates signals indicating the existence of a physical world separate from the virtual world (e.g. joystick, weight of wearable devices, and external noise); (2) Extensiveness refers to the number of sensory modalities accommodated; (3) Surrounding involves the visual presentation of the VR technology, to which the physical world is shut out (e.g. head-mounted display, surround projection, and computer screen); (4) Vividness corresponds to the fidelity and resolution with which the VR technology simulates the desired environment (e.g. visual information and functionality); and (5) Matching to whether the viewpoint of the VR technology is modified to match the user's perspective through motion capture.

Each one of the five criteria influences, but may not be the sole determinant of, the user's perceptual experience [7,8]. In this systematic review, we objectively classified VR technologies as low, moderate, or highly immersive based on the extent to which they met [7] criteria. When a study differed in the level of immersion across multiple aspects, we averaged across criteria to determine a global immersion rating. For example, if a VR technology met low criteria on one aspect, moderate criteria on three aspects, and high criteria on one aspect, it was classified as moderate immersion. The numerical score was calculated by converting the scores of each aspect into a numerical value (low = 1, moderate = 2, and high = 3) to estimate an overall mean score for each VR technology.

2.5. Risk of bias and quality assessment

Two reviewers independently appraised the included studies using the Downs and Black tool (1998). The tool contains 27 questions across five sections and provides both an overall score for study quality and a bias score. The five sections included (1) study quality – 10 items, (2) external validity – 3 items, (3) study bias – 7 items, (4) confounding and selection bias – 6 items, and (4) power of the study – 1 item. Items are scored from 0 (“no”) to 1 (“yes”), excepting item 5 (scores ranging from 0–2; 0 = no; 1 = partially, and 2 = yes) and item 27 (scores ranging from 0–5; 0 = No power calculation is provided; 3 = The power calculation is provided but the importance or impact of the difference between groups used in the calculation is unclear; 5 = The difference between groups is clearly defined as a clinically important difference.) Downs and Black total score ranges were given in corresponding quality levels: (1) excellent (≥26), (2) good (20–25), (3) fair (15–19), and (4) poor (≤14) [26]. Only randomized studies could achieve a quality level of excellent in accordance with the scoring methodology of the Downs and Black checklist.

2.6. Strategy for data synthesis

A critical analysis of the literature was performed based on (1) a descriptive numerical summary based on the characteristics of the studies, samples, diagnoses, and types of VR technologies; (2) level of immersion; and (3) risk of bias and quality assessment of the studies. Based on the information available, the suitability of a meta-analysis was considered.

3. Results

Of 404 studies, a total of 22 were included in the systematic review, for an aggregated sample of 564 individuals with NCDs participating in these studies (Fig. 1). Given the variability in study outcomes, it was not feasible to conduct a meta-analysis. Most of the included studies were conducted in individuals with NCDs due to stroke (27.3%) [[27], [28], [29], [30], [31], [32]], followed by mild cognitive impairment (22.7%) [[33], [34], [35], [36], [37]], and AD (13.6%) [[38], [39], [40]]. In addition, 13.6% of the 22 studies included samples with nonspecified NCD [[38], [39], [40]], and 22.7% included groups with suspected NCD (e.g. questionable dementia and presence of memory deficits), multiple sclerosis, or diagnosis not confirmed [[41], [42], [43], [44], [45]].

Fig. 1.

PRISMA flow diagram for virtual reality and neurocognitive disorders. Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

3.1. Clinical information

Information of each study including the objective, clinical characteristics of the sample, main outcomes, and results is presented in Table 1. The Mini-Mental State Examination [49] was the most widely used tool to determine the severity of the NCDs with a wide range in the scores (10–30), followed by the Montreal Cognitive Assessment [50] with scores ranging from 18 to 30 points. Outcome measures include traditional neuropsychological tests (e.g. Rey Auditory Verbal Learning Test and Trail Making Test), as well as measures of suitability and acceptability. In general, the results of the interventions indicate an improvement at different levels of cognition (e.g. memory, dual tasking, and visual attention). Secondarily, a reduction in psychological aspects was confirmed after VR interventions targeting cognition (e.g. reduction of anxiety, higher levels of well-being, and increased use of coping strategies). Only one study showed no change in cognitive outcomes after the VR intervention most likely related to the relatively short training period and lack of training specificity for improving cognitive performance [37]. The rest showed positive outcomes or the feasibility to deliver the intervention. However, most of the studies did not report effect sizes (see Table 1).

Table 1.

Clinical information, outcomes, and main conclusions of studies on VR and NCDs

Author	Objective	Sample size	Diagnosis	NCD severity	Outcome measures	Effect sizes	Conclusion
Blackman et al. [46]	To evaluate the validity and reliability of a computer-generated virtual environment.	38	Mild to moderate dementia (AD and vascular)	MMSE = 15–29	Navigability, legibility, safety, comfort, and well-being	Reported only for task performance (r = nonsignificant to −0.53) between real and virtual.	As impairment increases, the environment becomes more challenging for participants.
Davis et al. [38]	To examine the effect of salient visual cues on the wayfinding performance of older adults with and without AD.	88 (50 in the control condition, 38 with NCD)	MCI and AD	MMSE and MoCA: 25.87 (3.01) and 18.97 (3.58) for AD and MCI and 29.16 (0.99) and 25.64 (2.09) for healthy controls, respectively	Speed of navigation (time needed to find the destination as a measure of learning) and the number of goal acquisitions	∗ Cohen's d of 0.52 for goal acquisitions in the AD/MCI group	Salient cues increase navigation speed and goal acquisition in individuals with AD/MCI.
De Luca et al. [27]	To determine the effects of a VR intervention on cognitive function in patients who had stroke, as compared with traditional cognitive rehabilitation.	12	Ischemic or hemorrhagic stroke in the chronic stage	MMSE = 10–23	MoCA, frontal assessment battery, attentive matrices, and trial making test	Not reported and data insufficient to calculate it	BTs-Nirvana was successful at improving function in cognitive domains for patients who had stroke, relative to a control group.
De Luca et al. [28]	To evaluate the effects of combined conventional and VR rehabilitation techniques on cognitive functioning in an individual who had stroke.	1	Hemorrhagic stroke in the postacute phase	Not specified	Cognition, attention, anxiety, depression, coping strategies, and functional status	Not reported this as a case study.	Relaxation and respiratory techniques in a semi-immersive VR environment are superior to conventional relaxation and respiratory techniques in improving attention, coping strategies, and in reducing anxiety symptoms.
Delbroek et al. [33]	To investigate whether VR training improves cognitive, balance, and dual-task performance in older adults with MCI.	20	MCI	MoCA < 26	MoCA, the Dutch version of the Intrinsic Motivation Inventory (IMI), and the Observed Emotion Rating Scale (OERS)	∗ Cohen's d of 0.16 for MoCA	The VR technology was successful at improving balance and dual-tasking in older adults with MCI with no changes in global cognition.
Faria, et al. [29]	To observe the effect of VR rehabilitation on the functioning of individuals who had stroke.	18	Stroke	MMSE ≥ 15	Addenbrooke Cognitive Examination (primary outcomes) and the Trail Making Test A and B, Picture Arrangement from WAIS-III and Stroke Impact Scale 3.0 (secondary outcomes)	r's ranging from 0.01 to 0.85	Cognitive rehabilitation through an ecologically valid VR system has more impact than conventional methods.
Flynn et al. [47]	To examine the feasibility of VR technology for use by persons with dementia.	6	Dementia	Only one score known for one participant: MMSE = 12	Psychological well-being	Not reported and data insufficient to calculate it.	People with dementia have little difficulty in navigating VR environments and face no changes in psychological and physical well-being as a result of exposure to the VE.
Hwang & Lee [34]	To investigate the effect of VR on cognitive function and balance in elderly individuals with MCI.	24 (12 in experimental, 12 in control)	MCI	MMSE experimental = 22.4 ± 0.7. MMSE controls = 22.3 ± 0.7	Memory (Visual Span Test) and attention (WCT)	∗ Cohen's d between 0.14 and 0.5 in the experimental group	The VR program is one of the effective intervention methods for improving cognitive functions such as memory.
Kim et al. [30]	To investigate the effect of VR on cognitive recovery in individuals who had stroke.	28 (15 in experimental, 13 in control)	Stroke	MMSE experimental = 17.5 ± 3.9. MMSE controls = 16.4 ± 6.3	Computerized neuropsychological test (CNT, MaxMedica) and the Tower of London test	∗ Cohen's d between 0.48 and 0.84 in the experimental group	Visual attention and short-term visuospatial memory showed significant improvement in the VR group compared with the control group.
Kizony et al. [41]	To document the service delivery implemented by the VR system for people with ABI, over 2 years.	82 (74 with ABI, 8 with MS)	ABI or MS	MMSE ABI (n = 30) = 27.6 ± 1.9 (23–30); MoCA ABI (n = 39) = 25.8 ± 3.1 (18–30); MMSE MS (n = 2) = 29.0 ± 1.4 (28–30); MoCA MS (n = 6) = 28.5 ± 1.9 (25–30)	Trail Making Test A and B	Not reported and data insufficient to calculate it.	The significant improvements in the participants' scores on the TMT (entailing visuomotor scanning, divided attention, and cognitive flexibility) point to its effectiveness for a population that does not typically receive intensive therapy.
Lee et al. [31]	To explore patient-perceived difficulty and enjoyment during VR-based rehabilitation and the factors affecting those experiences.	8	Stroke	MMSE = 10–29	Levels of difficulty, enjoyment, and training intensity were assessed quantitatively with a visual analog scale.	Only provided for physical outcomes (Cohen's d between 0.50 and 0.96).	There was an enjoyment of the VR program when task difficulty and patient abilities were matched.
Man et al. [42]	To develop and implement VR-based memory training for older adults with dementia, and to examine the efficacy of the intervention on cognitive functions.	44 (20 in VR group, 24 in the therapist-led group)	Questionable dementia	VR group mean MMSE = 21.05, therapist-led group mean MMSE = 23	Multifactorial Memory Questionnaire and Fuld Object-Memory Evaluation	∗ Cohen's d between 0.03 and 2.2 in the experimental group	Nonimmersive VR participants showed greater improvements in objective memory performance as compared with a non-VR group, suggesting that VR may be a useful tool in memory rehabilitation.
Manera et al. [48]	To assess the acceptability, interest, and usability of VR in individuals with MCI.	57 (28 with MCI and 29 with dementia)	MCI or dementia	MMSE MCI = 25.4 ± 2.6. MMSE Dementia: 20.2 ± 3.1	Level of satisfaction, interest, discomfort, anxiety, the feeling of security, and fatigue	Not reported and data insufficient to calculate it.	Both participants with MCI and dementia were highly satisfied and interested in the attentional task, and reported high feelings of security and low discomfort, anxiety and fatigue.
Mirza & Yaqoob [35]	To observe the effects of VR cognitive training on cognition, blood pressure, and glucose levels in an individual with MCI.	1	MCI	MMSE = 23; MoCA = 24	General cognition, verbal fluency, and TMT scores	Not reported this as a case study.	VR successfully improved cognitive functioning in one individual with MCI.
Moyle et al. [39]	To measure and describe the effectiveness of a VR program on engagement, apathy, and mood states of people with dementia.	29 (10 individuals with dementia, 10 family members, and 10 staff members)	Dementia (Alzheimer = 7, Undisclosed = 3)	Psychogeriatric Assessment Scale = 13.21	Observed Emotion Rating Scale, Person–Environment Apathy Rating, and type of engagement	∗ Cohen's d between 0.08 and 1.5 in the group of individuals with dementia	Participants experienced more pleasure and a greater level of alertness.
Mrakic-Sposta et al. [36]	To evaluate the impact of an innovative-combined physical activity and cognitive training based on VR in individuals with MCI.	10	MCI	MMSE: 23.0 ± 3.4	Cognition (extensive neuropsychological battery: Attentional Matrices Test; RAVLT; ROCFT; TMT-A; Frontal Assessment Battery) and acceptability	∗ Cohen's d between 0.46 and 2.21 in the experimental group	The combined VR training protocol was able to effect MMSE tasks and to increase the global cognition levels of MCI.
Optale et al. [43]	To test the efficacy of a program of VR memory training in a group of rest-home residents with objective memory deficits.	36 (15 experimental, 16 control)	Presence of memory deficits	Memory deficits as documented by a corrected total score at the Verbal Story Recall Test below the cutoff value (15.76). MMSE experimental = 22.9 ± 5. MMSE controls = 20.99 ± 4.75	General cognitive abilities, verbal memory, executive functions, and visuospatial processing, and depression	Cohen's d between −0.33 and 0.75 for the training session and between −0.32 and 0.4 for the booster session.	The particular nature of the VR training may allow memory function training even with those affected by severe memory impairment.
Park & Yim [44]	To investigate whether a VR program could improve cognitive functioning and muscle strength and balance in community-dwelling elderly.	72 (36 experimental, 36 control)	Undiagnosed	MMSE experimental = 22.63 ± 4.91. MMSE controls = 22.88 ± 4.18	MoCA	∗Cohen's d of 0.43 in the experimental group	Physical activity via VR program decreased risked cognitive impairment.
Schwenk et al. [37]	To evaluate the feasibility and experience in using VR training in individuals with amnestic MCI.	22 (12 experimental, 10 control)	Amnestic MCI	MoCA score averaged 23.3 ± 2.6, experimental = 23.3 ± 3.1, control = 22.4 ± 3.0	MoCA and the Trail Making A and B tests	Only provided for physical outcomes (Balance indicators – Partial eta squared ranging from 0.213 to 0.257).	Lack of effect on cognitive performance most likely related to the relatively short training period and lack of training specificity for improving cognitive performance.
Threapleton et al. [32]	To explore the feasibility of using a VR intervention to support discharge after a stroke.	16 (intervention = 8, control = 8)	Stroke	NIHSS scale: minor stroke (1–4) 5 (31.25%); moderate stroke (5–15) 9 (56.25%); moderate to severe stroke (16–20) 1 (6.25%); severe stroke (21–42) 0 (0%); missing 1 (6.25%)	Health-related quality of life and a measure of satisfaction was obtained using the Patient Satisfaction Index.	∗ Cohen's d between 0.02 and 1.22 in the experimental group	It was feasible to recruit, randomize, and retain participants for follow-up assessments and to deliver the intervention to support discharge after stroke.
Vallejo et al. [45]	To determine which VR interface is more acceptable to an elderly population for rehabilitation purposes.	20	Undiagnosed (Dementia?)	Not reported	System Usability Scale	Not reported and data insufficient to calculate it.	To create a serious game based rehabilitation program, it is essential to take into account the usability of the involved devices, the person's abilities and also the motivations to play of the target population.
White & Moussavi [40]	To observe whether a VR navigation task can be used for neurocognitive treatment in an individual with Alzheimer's disease.	1	MCI vs. possible AD	MoCA = 24	MMSE and MoCA	Not reported this as a case study.	The VR treatment has shown some benefits for one person at an early stage of AD.

Abbreviations: VR, virtual reality; AD, Alzheimer's disease; MMSE, Mini-Mental State Examination; MCI, mild cognitive impairment; MoCA, Montreal Cognitive Assessment; WAIS, The Wechsler Adult Intelligence Scale; WCT, Word Color Test; ABI, acquired brain injury; MS, multiple sclerosis; CDR, Clinical Dementia Rating Scale; TMT, Trail Making Test; RAVLT, Rey Auditory Verbal Leaning Test; ROCFT, Rey-Osterrieth Complex Figure Test; NIHSS, National Institute of Health Stroke Scale.

^∗The effect size was not reported, but it was calculated based on data available.

3.2. VR technology and reported levels of immersion

Table 2 shows different VR levels of immersion as reported in the studies, the characteristics of the VR programs, and user experience. In regard to the description of the levels of immersion, as provided in the studies, 50% of the technologies were described as semi-immersive, 18.2% as immersive, 18.2% as nonimmersive, and 13.6% did not describe the level of immersion. The average number of sessions delivered for the 22 studies was 13.8 (SD = 14, range = 1–60). The average duration of a single session across the 22 studies was 31.4 minutes (SD = 11.3, range = 5–50). A total of 40.9% of the studies did not report any data regarding user acceptance. On average, the remaining studies indicated a good level of acceptance associated with enjoyment with the technology, user satisfaction, interest, engagement, motivation, safety, helpfulness, and easiness of use. Interestingly, adverse effects were not reported in 59.1% of the studies. Among the adverse effects reported, the most frequent ones included simulator sickness [38], negative emotions when participants fail in specific activities facilitated by the technology [33], oculomotor disturbances, nausea, and disorientation [47], neck pain [31], and some dizziness [44]. Only one study clearly indicated an absence of training-related adverse events [37]. The remaining studies indicated no adverse effects.

Table 2.

VR levels of immersion, characteristics of the VR programs, and user experience

Title	Name of VR application	Subjective level of immersion	Number of VR sessions and frequency	Length of each VR session	User acceptance	Adverse effects
Blackman et al. [46]	Not specified	Semi-immersive	Two sessions: (A) real-world and simulated VE walk and (B) adapted VE walk	Not specified	Partial because of technical difficulties (e.g. detailed resolution and reproduction of real-world peripheral vision and challenges to use the joystick).	Not reported
Davis et al. [38]	Virtual Senior Living	Semi-immersive	Each participant experienced 10 trials in each condition: 20 trials over two days.	30 minutes with a 15-minute break	Not specified	16 participants (control group n = 10, 19%; AD/MCI group n = 6, 12%) withdrew because of simulation sickness.
De Luca et al. [27]	BTs Nirvana	Semi-immersive	Three sessions weekly for two months	40 minutes	Not specified	Not reported
De Luca et al. [28]	BTs Nirvana	Semi-immersive	24 sessions, three times a week for 8 weeks	45 minutes	Not specified	Not reported
Delbroek et al. [33]	BioRescue	Semi-immersive	2 sessions a week for 6 weeks	Increasing length: 18 minutes in week one to 30 minutes in week 5	Participants found the VR training to be pleasant and useful for concentration, memory, and balance.	Overall, sadness, anger, and anxiety appeared only as a small reaction to the failure of an exercise.
Faria, et al. [29]	Reh@City	Not specified	12 sessions throughout 4 to 6 weeks	20 minutes per session	Good usability and satisfaction with the system with no reported problems using the interface.	Not reported
Flynn et al. [47]	VE BG with Systems Flybox joystick	Semi-immersive	Two sessions	50 minutes total, including 20 minutes per VR experience (2), and a 10–15 minute break	Participants perceived some objects as being realistic and moved naturally, felt in control of the simulation, and demonstrated the ease in using the joystick.	Oculomotor disturbances > nausea > disorientation
Hwang & Lee [34]	BioRescue	Not specified	20 sessions over 4 weeks	30 minutes	Not specified	Not reported
Kim et al. [30]	IREX	Not specified	3 times a week for 4 weeks	30 minutes	Participants found the system to be highly useable for rehabilitation and reported a high level of enjoyment.	Not reported
Kizony et al. [41]	CogniMotion	Semi-immersive	Twice a week for 2 months	30 minutes	Participants reported high levels of system usability and enjoyment.	Not reported
Lee et al. [31]	VR program	Semi-immersive	5–8 sessions, 3 days per week	20–30 minutes	Some participants with normal cognitive ability could not understand the rules, whereas participants with MCI could enjoy the game with only minimal assistance.	One participant said that she became reluctant to do the training because of pain in the back of her neck and flank while performing the training.
Man et al. [42]	VR program	Nonimmersive	10 sessions, over 2–3 times a week	30 minutes	Not specified	Not report
Manera et al. [48]	VR program	Nonimmersive	Single session	5 minutes	Participants reported a preference for the VR condition as compared with the paper condition. Low levels of anxiety, fatigue, and discomfort, and high levels of satisfaction, interest, and feelings of security with the VR program.	Difficulties in using the mouse to select the targets, and possibly due to eye strain from wearing the 3D glasses, and to the global VR setup, which was new to most of the participants.
Mirza & Yaqoob [35]	A combined aerobic-VR cognitive training program	Not specified	3 days a week for 12 weeks	30 minutes	Not specified	Not reported
Moyle et al. [39]	VR Forest	Immersive	Single session	15 minutes maximum, Mean = 10.22 (SD = 1.07)	The majority of participants reported a positive perception. The staff reported very few technical difficulties with setting up the technology.	50% of participants expressed significantly greater levels of anxiety/fear during the experience.
Mrakic-Sposta et al. [36]	VR program	Semi-immersive	3 sessions per week for 6 weeks	40–45 minutes	Participants enjoyed the VR program, with high levels of engagement and motivation.	Some reports of a slight sense of sickness.
Optale et al. [43]	VR memory training (VRMT)	Immersive	3 sessions per week for 3 months in the initial phase, 2 sessions a week for 3 months in the booster phase	30 minutes	Not specified	Not reported
Park & Yim [44]	VR kayak program	Semi-immersive	2 times a week for 6 weeks	20 minutes per session	Not specified	Some dizziness
Schwenk et al. [37]	A sensor-based balance training program	Semi-immersive	2 sessions a week for 4 weeks	45 minutes	Fun, safety, and helpfulness were reported.	No training-related adverse events occurred.
Threapleton et al. [32]	Virtual home	Nonimmersive	Single session	24 minutes in the feasibility study, 27 minutes in the pilot study	Feedback from the therapists indicated that the intervention was acceptable to them and identified further potential improvements for the content of the application.	Not reported
Vallejo et al. [45]	3D Memory Island	Immersive	Single session	Not specified	All the participants used this device easily and did not need any previous experience, but some had more difficulties with the touchpad.	Not reported
White & Moussavi [40]	VR navigation task	Immersive	3 sessions a week for 7 weeks	45 minutes per session	Not specified	Not reported

Abbreviations: VR, virtual reality; VE, virtual environment; MCI, mild cognitive impairment.

3.3. Objective level of immersion

Table 3 shows the objective levels of immersion based on the aforementioned criteria [7,8]. Most of the studies (40.9%) met the criteria for a moderate level of immersion, followed by 27.3% of high level of immersion, and 13.6 of low immersive experience. Surprisingly, the information about the immersive criteria allowing rating of the degree of immersion of the VR technology was not available in 18.2% of the studies.

Table 3.

Immersion levels of VR technologies

Author	Inclusiveness	Extensiveness	Surrounding	Vividness	Matching	Total score	Numerical score
Blackman et al. [46]	Low	Moderate	Moderate	Moderate	Low	Moderate	1.6
Davis et al. [38]	Low	Low	Low	Low	Low	Low	1
De Luca et al. [27]	Moderate	High	Moderate	High	High	High	2.6
De Luca et al. [28]	Moderate	High	Moderate	High	High	High	2.6
Delbroek et al. [33]	Low	Moderate	Low	Info unavailable	Moderate	Info unavailable	-
Faria, et al. [29]	Low	Low	Low	Low	Low	Low	1
Flynn et al. [47]	Low	Moderate	High	High	Low	Moderate	2
Hwang & Lee [34]	Info unavailable	Info unavailable	Info unavailable	Info unavailable	Info unavailable	Info unavailable
Kim et al. [30]	Low	Moderate	Low	Moderate	Moderate	Moderate	1.6
Kizony et al. [41]	Low	Moderate	Low	Moderate	Moderate	Moderate	1.6
Lee et al. [31]	Low	Moderate	Low	Moderate	High	Moderate	1.8
Man et al. [42]	Low	Low	Low	Low	Low	Low	1
Manera et al. [48]	Moderate	Moderate	Moderate	Moderate	Low	Moderate	1.8
Mirza & Yaqoob [35]	Low	Moderate	Info unavailable	Moderate	High	Info unavailable	-
Moyle et al. [39]	Low	High	Low	High	Moderate	Moderate	2
Mrakic-Sposta et al. [36]	Low	High	Moderate	Moderate	Moderate	Moderate	2
Optale et al. [43]	Low	Moderate	Low	Low	Moderate	Low	1.4
Park & Yim [44]	Moderate	Moderate	Moderate	Info Unavailable	Moderate	Info Unavailable	-
Schwenk et al. [37]	Low	Moderate	Low	Low	Moderate	Low	1.4
Threapleton et al. [32]	Low	Low	Low	Moderate	Low	Low	1.2
Vallejo et al. [45]	Low	Moderate	Low	Moderate	High	Moderate	1.8
White & Moussavi [40]	High	Moderate	High	Low	High	High	2.4

3.4. Risk of bias and quality assessment

A total quality score was calculated by rating the individual items in each of the five domains. The tool does not provide a cutoff score to classify the studies into either a low- or high-quality study, avoiding the artificial classification. The average quality score for the studies was 17.7 (SD = 4.1, range = 10–27), out of 31 possible points. We included case studies as part of this review. Table 4 presents the percentage of the studies that met the criteria in the checklist.

Table 4.

Percentages of studies meeting the criteria for quality assessment

Item	Question	Percent
1	Is the hypothesis/aim/objective of the study clearly described?	97.7
2	Are the main outcomes to be measured clearly described in the Introduction or Methods section?	90.1
3	Are the characteristics of the participants included in the study clearly described?	95.5
4	Are the interventions of interest clearly described?	95.4
5∗	Are the distributions of principal confounders in each group of subjects to be compared clearly described?	0 = 0 1 = 45.45 2 = 54.5
6	Are the main findings of the study clearly described?	97.7
7	Does the study provide estimates of the random variability in the data for the main outcomes?	72,735
8	Have all important adverse events that may be a consequence of the intervention been reported?	6.7
9	Have the characteristics of patients lost to follow-up been described?	77.3
10	Have actual probability values been reported for the main outcomes except where the probability value is less than 0.001?	75
11	Were the subjects asked to participate in the study representative of the entire population from which they were recruited?	47.7
12	Were those subjects who were prepared to participate representative of the entire population from which they were recruited?	45.5
13	Were the staff, places, and facilities where the patients were treated, representative of the treatment the majority of patients receive?	38.6
14	Was an attempt made to blind study subjects to the intervention they have received?	0
15	Was an attempt made to blind those measuring the main outcomes of the intervention?	20.5
16	If any of the results of the study were based on “data dredging” was this made clear?	88.6
17	In trials and cohort studies, do the analyses adjust for different lengths of follow-up of patients, or in case-control studies, is the time period between the intervention and outcome the same for cases and controls?	77.3
18	Were the statistical tests used to assess the main outcomes appropriate?	88.6
19	Was compliance with the intervention/s reliable?	72.7
20	Were the main outcome measures used accurate (valid and reliable)?	93.1
21	Were the patients in different intervention groups (trials and cohort studies) or were the cases and controls (case-control studies) recruited from the same population?	70.4
22	Were study subjects in different intervention groups (trials and cohort studies) or were the cases and controls (case-control studies) recruited over the same period of time?	61.4
23	Were study subjects randomized to intervention groups?	49.9
24	Was the randomized intervention assignment concealed from both patients and health care staff until recruitment was complete and irrevocable?	6.8
25	Was there adequate adjustment for confounding in the analyses from which the main findings were drawn?	25
26	Were losses of patients to follow-up taken into account?	79.5
27†	Did the study have sufficient power to detect a clinically important effect where the probability value for a difference being due to chance is less than 5%?	0 = 50 3 = 18.18 5 = 31.82

NOTE. (A) study quality – 10 items (items 1–10), (B) external validity – 3 items (items 11–13), (C) study bias – 7 items (items 14–20), (D) confounding and selection bias – 6 items (items 21–26), (E) power of the study – 1 item (item 27).

^∗0 = No; 1 = Partially, 2 = Yes.

^†0 = No power calculation is provided; 3 = The power calculation is provided, but the importance or impact of the difference between groups used in the calculation is unclear; 5 = The difference between groups is clearly defined as a clinically important difference.

The analysis of Reporting, which assessed whether the information provided in the article was sufficient to allow a reader to make an unbiased assessment of the findings of the study, revealed that the distributions of principal confounders in each group of participants to be compared were not clearly described and that all important adverse events that may be a consequence of the intervention have not been reported. On average, reporting scores were high (7.8 of 10, SD = 1.6, range = 5–10).

External validity, addressing the extent to which the findings from the studies could be generalized to the population from which the study participants were derived, indicates that the staff, places, and facilities where the participants were treated were not representative of the treatment the majority of clients receive. In almost 40%, the interventions were delivered in a university laboratory, which makes sense given that these are new treatments and that some of them correspond to feasibility studies. On average, external validity scores were high (1.3 of 3, SD = 1.1, range = 0–3).

The analysis of Bias, which addressed biases in the measurement of the intervention and the outcome, revealed that there was not an attempt made to blind study participants to the intervention they have received. On average, Bias scores were high (4.4 of 7, SD = 1, range = 2–6).

Confounding, that addressed bias in the selection of study participants revealed that there was not adequate adjustment for confounding in the analyses from which the main findings were drawn. However, average confounding scores were moderate (2.9 of 6, SD = 1.4, range = 0–5).

Finally, the item evaluating Power, or whether the negative findings from a study could be due to chance indicated that only 31.8% of the studies had sufficient power to detect a clinically important effect where the probability value for a difference being due to chance is less than 5%. In addition, average Power scores were low (1.2 of 5, SD = 1.9, range = 0–5).

Based on the criteria to classify Downs and Black score ranges [26], none of the studies reached excellent quality levels (≥26). A total of 10 studies reached good quality levels (20–25) [27,29,30,[32], [33], [34],38,39,41,47], seven reached fair quality levels (15–19) [31,36,37,[42], [43], [44],48], and only five reached poor quality levels (≤14) [28,35,40,45,46].

4. Discussion

The present systematic review was conducted to study the effects of virtual reality on overall cognitive and psychological rehabilitation in individuals with NCDs. We provided descriptive analyses based on the characteristics of the studies, study samples, diagnoses, types of VR technologies, subjective and objective levels of immersion, and the risk of bias and quality assessment of 22 studies with an aggregated sample of 564 individuals with NCD. We extracted the information with two independent reviewers to increase the accuracy in the process.

The main finding is that the results of the VR interventions in individuals with NCDs indicate an improvement at different levels of cognition (e.g. memory, dual tasking, and visual attention) and an improvement in psychological functioning (e.g. reduction of anxiety, higher levels of well-being, and increased use of coping strategies). VR interventions have been mainly used in individuals with NCDs due to stroke, followed by individuals with mild cognitive impairment and AD. The evidence provided in this systematic review supports the use of VR interventions as a way to provide cognitive rehabilitation and to treat psychological symptoms in individuals with NCDs of different etiologies.

Most of the VR technologies reported subjective levels of immersion described as semi-immersive. The average number of sessions was close to 14, with a duration of approximately 30 minutes each, two or three times per week. As such, VR interventions were primarily delivered in university laboratories or hospital facilities as part of innovation initiatives. Although almost 40% of the studies did not report acceptability, VR interventions seem to be greatly accepted by individuals with NCDs (e.g. enjoyment with the technology, user satisfaction, interest, engagement, motivation, safety, helpfulness, and easiness of use). Adverse effects do not seem to be exclusive of NCDs (e.g. simulator sickness, negative emotions when participants fail in specific activities facilitated by the technology, oculomotor disturbances, nausea, disorientation, neck pain, and some dizziness), as they have been reported in only 5% of participants in a study of VR aiming to document those effects [51]. As such, adverse effects need to be monitored but do not represent a reason for exclusion on their own.

Most of the studies met objective criteria for moderate levels of immersion. Higher levels of immersion can increase the user experience and play a major role in the sense of presence. The sense of presence corresponds to the experience of actually feeling “being there,” as in a real-world situation, which is vital to substantially affect the behavioral responses [17]. In the present state of the literature, we do not know if a fully immersive VR technology is better than a moderate or low immersive VR technology in individuals with NCDs.

Although the overall quality of the studies was good, it is important to insist on the documentation of adverse effects, make sure to attempt to blind study participants to the intervention they receive, and make sure to adjust for confounding in the analyses from which the main findings are drawn and increase power. In regard to the external validity, the results of this systematic review indicate that the staff, places, and facilities where the participants were treated were not representative of the treatment the majority of clients receive. This was expected as VR technologies are being developed, and their use is in the early stages. There were no studies that could achieve a quality level of excellent in accordance with the scoring methodology of the Downs and Black checklist. As such, future research needs to focus on randomized controlled studies.

5. Conclusion and recommendations for research and practice

In conclusion, we presented a systematic review of the literature on VR technologies for the rehabilitation of cognition and psychological functioning in minor and major NCDs. VR interventions are useful to improve cognition (e.g. memory, dual tasking, and visual attention) and psychological functioning (e.g. reduction of anxiety, higher levels of well-being, and increased use of coping strategies) in individuals with NCD. VR interventions are mainly used in individuals with NCDs due to stroke. Although this systematic review was not suitable for a meta-analysis, the results indicate that VR interventions are helpful and promising in providing cognitive rehabilitation and to treat psychological symptoms in individuals with NCDs of different etiologies, which should be further explored in future with well-designed studies.

Based on the conclusions of the present review, the following recommendations for future studies can help to increase research in individuals with NCDs.

• 1.To describe the clinical information of the samples. The type and severity of NCD are very important to be able to compare findings across studies. The use of widely known instruments to track cognitive changes (e.g. MMSE and MoCA) and a clear diagnosis allow clinicians to easily identify candidates for these interventions.
• 2.To conduct longitudinal studies on VR interventions of NCDs. Longitudinal studies allow capturing the progression of NCDs and help to observe whether the benefits of the VR interventions are maintained with the evolution of the disease.
• 3.To compare different levels of immersion. To date, we do not know whether higher levels of immersion are more effective to treat cognitive and psychological problems in individuals with NCDs, as compared to lower levels of immersion.
• 4.To systematically assess user acceptance and adverse effects. We do not know whether adverse effects or acceptance levels differ in NCDs of different etiologies or in the same NCD at different stages of the disease.
• 5.To use widely known measures for cognition and psychological outcomes showing cross-cultural validity. The use of core instruments that have been developed and used in different cultures will allow performing a meta-analysis and facilitate comparisons across studies.
• 6.Report effect sizes for all the outcomes. This will allow readers to have a better appreciation of the effect of VR interventions on cognition.
• 7.Provide examples of generalization to real-life situations. This could provide evidence that the results of VR interventions have an impact on the everyday life of individuals with NCDs. Measures can be collected through family caregiver reports or professional caregivers.

Research in context.

• 1.Systematic review: The authors conducted a systematic review of the published literature on immersive and nonimmersive virtual reality technologies targeting cognition in minor and major neurocognitive disorders following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.
• 2.Interpretation: Virtual reality interventions are useful to improve cognition (e.g. memory, dual tasking, and visual attention) and psychological functioning (e.g. reduction of anxiety, higher levels of well-being, increased use of coping strategies) in individuals with minor and major neurocognitive disorders.
• 3.Future directions: To improve research in this area, we recommend to describe the clinical information of the samples, to conduct longitudinal studies on virtual reality interventions of neurocognitive disorders, to compare different levels of immersion, to systematically assess user acceptance and adverse effects, to use widely known measures for cognition and psychological outcomes showing cross-cultural validity, to report effect sizes for all the outcomes, and to provide examples of generalization to real-life situations.