Skip to main content
JMIR Serious Games logoLink to JMIR Serious Games
. 2022 Feb 21;10(1):e32117. doi: 10.2196/32117

Immersive Virtual Reality–Based Cognitive Intervention for the Improvement of Cognitive Function, Depression, and Perceived Stress in Older Adults With Mild Cognitive Impairment and Mild Dementia: Pilot Pre-Post Study

KaiYan Zhu 1, QiongYao Zhang 2, BingWei He 3, MeiZhen Huang 1, Rong Lin 1,4, Hong Li 1,4,
Editor: Nabil Zary
Reviewed by: Nathan Martin, Ludmiła Lamparska
PMCID: PMC8902670  PMID: 35188466

Abstract

Background

The incidence of dementia is increasing annually, resulting in varying degrees of adverse effects for individuals, families, and society. With the continuous development of computer information technology, cognitive interventions are constantly evolving. The use of immersive virtual reality (IVR) as a cognitive intervention for older adults with mild cognitive impairment (MCI) and mild dementia (MD) is promising, although only few studies have focused on its use.

Objective

The Chinese virtual supermarket (CVSM) IVR system was developed to provide a comprehensive and individual cognitive intervention program for older patients with MCI and MD. The aim of this study was to explore the feasibility and clinical effectiveness of this 5-week IVR-based cognitive intervention.

Methods

A pretest-posttest study design was conducted with 31 older adults with MCI and MD from August 2020 to January 2021. All participants participated in a 5-week immersive virtual cognitive training program using the CVSM system. Feasibility was assessed as the incidence and severity of cybersickness symptoms and participant satisfaction based on questionnaires conducted after the intervention. Clinical effectiveness was evaluated using neuropsychological assessments, including several commonly used measures of cognitive function, depression, perceived stress, and activities of daily living. Measurements were obtained at baseline and after the intervention period.

Results

A total of 18 patients with MCI (mean age 82.94 [SD 5.44] years; 12 females) and 13 patients with MD (mean age 85.7 [SD 4.67] years, 10 females) participated in this pilot study. Both groups showed significant improvements in all cognitive function measurements (P<.001). The MD group had a significantly greater improvement in general cognitive function compared to the MCI group in Montreal Cognitive Assessment Scale, Symbol Digit Modalities Test, Shape Trail Test, and Auditory Verbal Learning Test. Furthermore, an intervention effect was observed in the improvement of perceived stress (P=.048 for MD group, P=.03 for MCI group ).

Conclusions

The use of the CVSM system may be effective in enhancing the cognitive function of patients with MCI and MD, including general cognitive function, memory, executive function, and attention. IVR technology enriches cognitive intervention approaches and provides acceptable, professional, personalized, and interesting cognitive training for older adults with cognitive impairment.

Trial Registration

ClinicalTrials ChiCTR2100043753; https://trialsearch.who.int/Trial2.aspx?TrialID=ChiCTR2100043753

Keywords: cognitive intervention, dementia, immersive virtual reality, mild cognitive impairment, cognitive impairment, virtual reality, VR, older patients, usability, memory, stress

Introduction

China is rapidly transforming into an aging nation. Based on the seventh national survey conducted by the National Bureau of Statistics, the number of older adults (aged ≥60 years) in China was 264.01 million (18.7% of the total population) at the end of 2020, including 190.63 million individuals aged ≥65 years (13.5% of the total population). It is predicted that there will be 400 million Chinese citizens aged ≥65 years by 2050, including 150 million aged ≥80 years [1]. Population aging is accompanied by an increased prevalence of mild cognitive impairment (MCI) and mild dementia (MD). In China, the prevalence of MCI in individuals ≥65 years is 10%-20%, and over 50% of these patients progress to dementia within 5 years. The incidence of dementia among individuals ≥60 years of age in China is predicted to increase from 14% in 2015 to 33% by 2050 [2,3]. The high prevalence of dementia in the older population is a significant social and economic burden on patients, patient families, and the existing health care system in China. The total cost associated with dementia is increasing at an alarming rate in China and is expected to exceed US $9.12 trillion by 2050 [4].

Although several studies regarding pharmacological treatments for MCI and MD have been conducted, the effectiveness of pharmacological treatments is limited. Therefore, safe treatment alternatives for MCI and MD have been developed, including cognitive interventions [5]. Cognitive training is among the most frequently used cognitive interventions and typically involves the repeated practice of a set of structured tasks [6,7]. In recent decades, many different cognitive training approaches aimed at improving and maintaining cognitive ability have been developed to reduce the progression of MCI and MD. However, most traditional cognitive training programs conducted by a neuropsychologist are based on face-to-face exercises [8,9], requiring the identification of a convenient meeting location, the coordination of schedules, and a dedicated training time. With the rapid development of computer science and technology, cognitive training exercises can now be delivered via computers or mobile technology, resulting in better patient compliance owing to the convenience of cognitive training [10-12]. Therefore, an increasing number of studies have focused on cognitive interventions based on computer technology.

Virtual reality (VR) is a new computer technology that was created with the development of multimedia technology and used in military science before being applied to the medical sciences [13]. VR is a computer-generated effect that can simulate a given scene through 3D graphics and other sensory experiences (vision, touch, and motion feeling) [14] by using special electronic devices such as computer keyboards, computer mice, speech/voice recognition, motion sensors, and haptic devices [15]. VR can be divided into 2 types according to the degree of immersion: nonimmersive and immersive virtual reality (IVR). The nonimmersive system is a desktop-based VR with low interaction (such as with a keyboard and joypad). IVR is characterized by the use of more interactive tools, including a head-mounted display or a cave automatic virtual environment, which allows patients to interact with such a virtual environment from the first-person perspective [16]. Nonpharmacological interventions based on IVR have been applied in clinical settings for pediatric patients, patients with psychotic disorders, older adults, and patients with MCI and MD [17-21]. Meta-analyses and systematic reviews have reported the effectiveness and advantages of VR for patients with MCI and MD. A recent systematic review reported that IVR has a potentially positive effect on global cognitive function, attention, and emotion [22]. Compared to traditional cognitive training, the VR environment is highly flexible and allows for cognitive training in environments that are either impossible or unsafe in real life. For example, IVR enables older adults with limited physical abilities to go shopping [23], ride a bicycle in a city [24], and cook [25] in a safe environment. In addition, the flexibility of VR allows for the adjustment of various parameters such as the duration of recall, types of stimuli [26], and numbers or similarities of distractors [27], ensuring that the settings match the patient’s individual capabilities.

Although IVR in the clinical setting has many advantages, the feasibility of its use among older adults remains controversial. A mixed method pilot study (n=10) examined the effects of a 15-minute interactive IVR forest experience on the level of engagement, apathy, and mood states of people with dementia and found that the use of IVR brought pleasure to participants but also increased their levels of fear and anxiety [28]. In addition, the extent to which IVR-based cognitive training can benefit cognition in patients diagnosed with MCI or MD is unclear. Systematic reviews have also reported mixed results [29-31]. Therefore, more research regarding IVR-based cognitive training for patients with MCI and MD is needed. In this study, an IVR-based cognitive training program, the Chinese virtual supermarket (CVSM), was developed for use in older adults with MCI and MD, and a pretest-posttest study was conducted to evaluate the effects on neuropsychological outcomes in older adults with MCI and MD. We hypothesized that after the 5-week IVR-based cognitive training session, patients with MCI and MD would have significantly improved cognitive function and other health-related outcomes.

Methods

Recruitment

The proposed virtual cognitive assessment platform was installed for clinical testing at the Fujian Good Health Care Center. Older adults were recruited for the study via popular science lectures on VR, virtual game demonstrations, and VR experience activities. The participants were screened according to the inclusion and exclusion criteria. This study included 35 participants (21 with MCI and 14 with MD). All participants were ≥60 years of age, had normal or corrected-to-normal vision, and could communicate in Mandarin Chinese. Participants with MCI were diagnosed according to the Peterson diagnostic criteria [32]: clinically confirmed memory loss, intact or slightly impaired activities of daily living (score <23 for patients <75 years of age and <25 for patients >75 years of age), cognitive impairment based on the Chinese-Changsha version of the Montreal Cognitive Assessment Scale (MoCA) (abnormal value: MoCA score <13 for illiterate individuals, <19 for individuals with a primary school diploma, and <24 for individuals with a junior high school diploma or above), and preserved general cognitive function assessed using the Mini-Mental State Examination (MMSE) (24-30 points). Participants with MD were diagnosed by an experiential psychiatrist according to the International Classification of Diseases 10th revision [33] with a Clinical Dementia Rating score ≤1. Participants with severe audiovisual impairments (deafness, cataracts, or glaucoma), psychiatric and logic disorders (bipolar disorder, schizophrenia, stroke, Parkinson disease, or epilepsy), or a history of taking drugs affecting cognitive function in the 6 months prior to this study were excluded. All participants provided written informed consent for their participation in this study. This study was approved by the ethics committee of Fujian Provincial Hospital (K2020-06-006).

Intervention Instruments

Virtual Environment

The CVSM has applied for national computer software copyright, registration 2021SR0993516, which was developed by Fujian Provincial Hospital in collaboration with the School of Mechanical Design Manufacturing and Automation, Fuzhou University, and the Rhino Technology Group. The virtual environment of the CVSM was developed and rendered using a Unity 3D engine and was run on a Dell Precision T3600 PC with a CPU Intel I5-6400 processor and a GTX 1600 graphics card. The system required a capacity of 277 MB. The 3D virtual supermarket was presented using HTC VIVE Pro Eye, which provided a stereoscopic vision via 2 screens in front of the eyes (the resolution of the binocular combination was 2160×1200 pixels). The HTC Pro Eye allowed the participant to rotate his or her head for a 360° view of the virtual scene and to interact and walk freely in the virtual environment. With the help of an assistant, participants were asked to stand in an open room wearing a VR helmet (Figure 1). Through the headset, the participant viewed a rectangular virtual supermarket (9.8 m×18 m). Participants used a wireless remote control to select items within the virtual scene.

Figure 1.

Figure 1

The Chinese virtual supermarket system.

IVR-Based Cognitive Intervention

The CVSM system was designed as a simple, easy-to-learn, personalized immersive VR cognitive management program for older adults and was divided into 3 modules: an operation learning module, a cognitive evaluation module, and a cognitive intervention module. In the operation learning module, participants were taught how to wear the 3D VR helmets and operate VR handles to interact with the virtual supermarket during 3 separate 10- to 15-minute sessions. Multiple exercises were used to help the participants adapt to the virtual environment. In the cognitive evaluation and cognitive intervention modules, the participants were presented with a list of 3-12 familiar virtual images and tags of common items (such as oranges or toothpaste) that they were asked to memorize within a minute and a half (Figure 2). After memorizing the list, the participants engaged in a number sorting game for 20 seconds (Figure 3) before entering the supermarket to buy the items on the memorized list. Then, participants were asked to locate the counter and pay in RMB bills, as shown in Figure 4. The final screen in the game confirmed that the payment was completed, and a report listing the quantities of products purchased, nonshopping list items, and the total time needed for completion was displayed. The participant’s task completion in the cognitive assessment module was set as the initial difficulty level in their cognitive training. The CVSM included 12 levels of difficulty according to memory quantity, delay time, calculation difficulty, and interval retrieval times. After completing 3 shopping sessions that included a maximum of 1 error each, the participant could increase his or her level of difficulty. Participants were instructed to train 3 times per week over a 5-week period and to complete 3 training sessions on each training day, with each session lasting 20-30 minutes. The entire cognitive training program ranges from simple to complex, involving training in multiple cognitive domains such as memory, attention, executive function, and calculation ability.

Figure 2.

Figure 2

The shopping list of the Chinese virtual supermarket system.

Figure 3.

Figure 3

The number sorting game of the Chinese virtual supermarket system.

Figure 4.

Figure 4

The payment interface of the Chinese virtual supermarket system.

Instruments to Measure Outcomes

A pretest-posttest study design was used to evaluate the effects of the VR-based cognitive intervention using the CVSM system in older adults with MCI and MD. Various measurements were obtained to evaluate feasibility and effectiveness of the IVR intervention. Demographic characteristics such as age, sex, and years of education were recorded during the initial evaluation. Trained psychologists assessed the participants.

Feasibility

Feasibility was assessed as the incidence and severity of cybersickness symptoms and participant satisfaction based on the answers provided in a questionnaire conducted after the intervention. The questionnaire was developed specifically for this study.

Severity of Cybersickness Symptoms

All participants were assessed for the severity of IVR by using the Simulator Sickness Questionnaire, which consists of 16 items regarding 3 factors: nausea (sweating, difficulty concentrating, or stomach awareness), oculomotor disturbance (headache, eyestrain, or blurred vision), and disorientation (head fullness, dizziness with eyes opened or closed, or vertigo). The severity of symptoms was categorized as none, mild, moderate, or severe. The Simulator Sickness Questionnaire score was calculated as the sum of the scores of the 16 items multiplied by 3.74, and it ranged from 0 to 179.52 [34].

Incidence of Cybersickness Symptoms

The incidence of cybersickness symptoms was calculated.

Satisfaction

The satisfaction questionnaire was used to assess the views and opinions of the participants regarding the intervention. Items were rated on a 4-point Likert scale (4=strongly agree, 3=agree, 2=neutral, and 1=poor). The total score ranged from 8 to 32, with higher scores indicating higher satisfaction. The causes of patient satisfaction were further investigated via open interviews.

Effectiveness

In this pilot study, the neuropsychological evaluation included several commonly used measures of cognitive function, depression, and perceived stress.

Cognitive Function

Global cognition was measured using the MMSE and MoCA, which have both been shown to have high test-retest reliability and be sensitive to changes in people with MD and MCI. MMSE and MoCA are respectively composed of 30 cognitive domain-related problems such as attention, language, word recall, time, and place positioning. The higher the score, the higher the cognitive level [35,36]. The effects of the treatments on memory function, executive function, and attention were assessed using the Auditory Verbal Learning Test (AVLT), the Shape Trail Test (STT), and the Symbol Digit Modalities Test (SDMT), respectively. In this study, the AVLT will be used to evaluate language learning and memory function, which includes 3 tests: immediate recall, short-term delayed recall, and long-term delayed recall [37]. The score of AVLT in this study is the sum of the 3 test scores. The higher the total recall score, the better the memory function. The STT was developed by Agnes Chan from the Chinese University of Hong Kong to evaluate executive function, and it consists of 2 components. The STT-A consists of 25 consecutive numbers—from 1 to 25. The STT-B was modified as 25 numbers enclosed in 13 circles (from 1 to 13) and 12 squares (from 1 to 12). Taking the sum of completion time as the scoring standard, the shorter the time, the better the executive function [38].The SDMT is used to assess the tester’s ability of attention, learning, conversion, and involves a substitution task using a coding key with 9 different abstract symbols, each paired with a numeral. Below the key, a series of these symbols was presented and the participant was asked to write down the corresponding number for each symbol. Participants need to complete as many of 115 items as they could in 90 seconds. The number of correct substitutions within this time was recorded as their score [39].

Depression

The Geriatric Depression Scale (GDS) is a self-report evaluation. The scale comprises 30 items: 10 items confirm depression if the answers are negative, while the remaining 20 items confirm depression if the answers are positive. Normal scores range from 0 to 10; a score of 11-20 indicates mild depression and 21-30 indicates moderate and severe depression [40].

Perceived Stress

The Chinese Perceived Stress Scale (PSS) is a scale for assessing individual stress. The scale mainly includes 14 self-assessment items: The 4th to 7th, 9th to 10th, and 13th items are used reverse score. The 1st to 3rd, 8th, 11th to 12th, and 14th items adopt positive scoring. The score of Chinese PSS equals total score minus 14. A higher score indicates that psychological pressure is higher, which is harmful to the physical and mental health [41].

Statistical Analysis

Baseline data are presented using mean and standard deviation. Student t test and Wilcoxon signed rank test were used to analyze the results to evaluate the effect of the VR cognitive intervention in patients with MCI and MD. The P value was corrected by false discovery rate to reduce the risk of type I error. In order to determine the effective size of the results, the effect size calculator (Cohen d) for ttest was used. The data were analyzed according to the intention-to-treat principle. Missing data were replaced with the individual's available data at baseline or were obtained by phone. All statistical analyses were conducted using the SPSS software (version 22.0; IBM Corp). Statistical significance was set at P<.05.

Results

Participant Characteristics

A total of 52 older adults showed interest in participating in this pilot study and were screened for eligibility; 17 participated in the upgrading and transformation of the CVSM system in the preparation stage and 35 agreed to participate in the feasibility study. Four participants (1 with MD and 3 with MCI) were unable to complete the study owing to physical limitations. Therefore, 18 participants with MCI (12 females and 6 males; mean age 82.94 [SD 6.44] years; mean years of education 11.0 [SD 3.97] years) and 13 participants with MD (10 females and 3 males; mean age 85.7 [SD 4.67] years; mean years of education 11.23 [SD 4.71] years) finally completed all courses of VR-based cognitive training as planned and were included in the final analysis. There were no significant differences between the MCI and MD groups at baseline with the exception of cognitive function (Table 1).

Table 1.

Participants’ baseline characteristics (N=31).

Characteristic MCIa (n=18) MDb (n=13) P value
Sociodemographic data

Age (years), mean (SD) 82.94 (6.44) 85.76 (4.67) .19

Female, n (%) 12 (67) 10 (77) .83

Education (years), mean (SD) 11.00 (3.97) 11.23 (4.71) .88
Cognitive function, mean (SD)

MoCA-CSc 21.56 (3.31) 13.92 (4.68) <.001

MMSEd 26.06 (2.99) 18.54 (5.73) <.001

AVLTe 38.11 (10.76) 25.38 (10.12) .002

STTf 352.72 (132.79) 523.23 (307.71) .04

SDMTg 21.83 (11.04) 12.85 (9.56) .03
Psychosocial measures, mean (SD)

PSSh 14.56 (12.05) 19.23 (15.28) .35

GDSi 5.11 (4.23) 5.00 (4.63) .95

aMCI: mild cognitive impairment.

bMD: mild dementia.

cMoCA-CS: Chinese-Changsha version of the Montreal Cognitive Assessment Scale.

dMMSE: Mini-Mental State Examination.

eAVLT: Auditory Verbal Learning Test.

fSTT: Shape Trail Test.

gSDMT: Symbol Digit Modalities Test.

hPSS: Perceived Stress Scale.

iGDS: Geriatric Depression Scale.

Cybersickness Symptoms

Eight participants had mild cybersickness symptoms in the first 4 activities (range of scores: 3.74-11.22 points). After 3 consecutive activities, the probability of the development of simulator disease was significantly reduced (Figure 5). After the fifth intervention, there were no reports of simulator disease. No adverse events such as injuries, falls, or quarrels were reported during the IVR interventions.

Figure 5.

Figure 5

Frequency of cybersickness symptoms.

Effectiveness of the Intervention

Statistical analysis was performed using the SPSS version 22.0. For comparison of the continuity variables of normal distribution, Student t test was used (Table 2). Otherwise, Wilcoxon signed rank test was used (Table 3). In order to reduce the risk of type I error, the P value was corrected by false discovery rate. As seen in Table 2 and Table 3, there is a significant difference between average pretest and posttest results in both groups (q<0.05), except GDS and PSS. In order to calculate the effect size of the results, Cohen d has been used. The effect size of the MD group results is significantly higher compared with that of the MCI group in MoCA, SDMT, STT, AVLT, and GDS.

Table 2.

Statistical analysis of preintervention and postintervention results of mild cognitive impairment and mild dementia groups by using Student t test and effect size calculation results with Cohen d.

Group, index Preintervention, mean (SD) Postintervention, mean (SD) t value (df) P value q value Cohen d
Mild dementia

MoCA-CSa 13.92 (4.68) 21 (3.89) –9.826 (12) <.001 0.017 1.64

MMSEb 18.54 (5.74) 24.85 (5.74) –7.697 (12) <.001 0.017 1.09

SDMTc 12.85 (9.56) 17.31 (7.74) –3.603 (12) .004 0.009 0.51

PSSd 19.23 (15.28) 13.15 (10.95) 2.205 (12) .048 0.056 0.45
Mild cognitive impairment

MoCA-CS 21.56 (3.31) 25.67 (2.93) –7.857 (17) <.001 0.023 1.31

STTe 352.72 (132.79) 254 (66.18) 4.51 (17) <.001 0.023 0.68

SDMT 21.83 (11.04) 27.22 (11.97) –5.422 (17) <.001 0.023 0.46

aMoCA-CS: Chinese-Changsha version of the Montreal Cognitive Assessment Scale.

bMMSE: Mini-Mental State Examination.

cSDMT: Symbol Digit Modalities Test.

dPSS: Perceived Stress Scale.

eSTT: Shape Trail Test.

Table 3.

Statistical analysis of preintervention and postintervention results of mild cognitive impairment and mild dementia groups by using Wilcoxon test and effect size calculation results with Cohen d.

Group, index Preintervention, mean (SD) Preintervention, median (min-max) Postintervention, mean (SD) Postintervention, median (min-max) Z value P value q value Cohen d
Mild dementia

AVLTa 25.38 (10.12) 24 (8-50) 38.53 (8.63) 38 (28-57) 2.97 .003 0.010 1.39

STTb 523.23 (307.71) 394 (227-1246) 338.76 (112.30) 306 (204-591) –3.04 .002 0.014 0.79

GDSc 5.00 (4.63) 4 (0-16) 3.38 (3.52) 3 (0-12) 1.42 .15 0.154 0.39
Mild cognitive impairment

MMSEd 26.06 (2.99) 26.5 (16-29) 29.06 (0.72) 29 (28-30) 3.42 .001 0.003 1.37

AVLT 38.11 (10.76) 36 (14-64) 51.33 (8.28) 51.5 (32-68) 3.72 <.001 0.007 1.38

PSSe 14.56 (12.05) 8.5 (0-40) 9.11 (12.11) 3 (0-38) 2.18 .03 0.033 0.45

GDS 5.11 (5.24) 3.5 (0-20) 4.22 (6.31) 2 (0-24) 1.16 .25 0.245 0.15

aAVLT: Auditory Verbal Learning Test.

bSTT: Shape Trail Test.

cGDS: Geriatric Depression Scale.

dMMSE: Mini-Mental State Examination.

ePSS: Perceived Stress Scale.

Satisfaction

The CVSM program received high user satisfaction ratings for all items (Table 4). All scores for cognitive intervention method/frequency, virtual environment, memory task, and interactive devices/modes were >3.50, indicating that the intervention was acceptable and positive. There were no complaints regarding the difficulty of the memory task. The visual and auditory supports (products, shopping lists, and hints) were reported to be clear and useful. The frequency of the intervention, comfort of the VR equipment, and difficulty of the operation received some neutral and poor ratings. One patient in the MCI group reported that the VR helmet was heavy, resulting in slight discomfort. Another patient in the MCI group reported that he did not like shopping in real life and was therefore neutral toward the VR supermarket scene. This patient also reported that the intervention was too frequent. Two patients in the MD group expressed concern about learning how to operate the IVR system, although they were able to complete the virtual shopping task with the guidance of the researchers.

Table 4.

Participant satisfaction (N=31).

Items Strongly agree, n (%) Agree, n (%) Neutral/poor, n (%) Mean (SD)
Q1. Are you satisfied with the method of the IVRa
intervention?
21 (68) 10 (32) 0 (0) 3.68 (0.47)
Q2. Are you satisfied with the frequency of the intervention? 23 (74) 7 (23) 1 (3) 3.71 (0.52)
Q3. Did you enjoy doing the shopping? 18 (58) 12 (39) 1 (3) 3.55 (0.56)
Q4. Did you recognize all the products in VRb supermarket? 31 (100) 0 (0) 0 (0) 4.00 (0)
Q5. Did you feel a sense of accomplishment after completing the shopping task? 17 (55) 14 (45) 0 (0) 3.55 (0.50)
Q6. Was it comfortable to wear the VR helmet? 25 (81) 5 (16) 1 (3) 3.77 (0.49)
Q7. Was it easy to learn VR handle operation? 26 (84) 3 (10) 2 (7) 3.74 (0.67)
Q8. Did you have a positive experience in using IVR? 25 (81) 6 (19) 0 (0) 3.81 (0.40)

aIVR: immersive virtual reality.

bVR: virtual reality.

Discussion

Principal Results

This study investigated the feasibility and effectiveness of a 5-week program for improving cognitive function in older adults with MCI and MD based on the increased use of IVR-based nonpharmacological interventions in the fields of cognitive impairment and geriatric preventive medicine. This is the first study to develop, implement, and evaluate the effects of an IVR-based cognitive training program in China. All participants who participated in the CVSM had improved executive function, attention, memory, and general cognitive function compared to baseline, which is consistent with the results of a meta-analysis of IVR-based cognitive training [42]. VR interventions are useful for patients with MCI or MD. Some studies [42,43] have reported limited positive effects in patients with MD and that the effects of VR were greater in patients with MCI than in patients with MD. However, in this study, the MD group had significantly greater improvement in general cognitive function than the MCI group. This difference in results may be due to the personalized difficulty setting of the CVSM system. An appropriate task difficulty setting and motivational incentives decrease the frustration of patients with MD when working on difficult tasks, enhancing the benefits of activated brain function [44]. Therefore, adjusting the appropriate task difficulty according to the basic cognitive level of patients with MD has a positive impact on improving the cognitive function of patients with MD.

The results of this study indicate that both groups had significantly improved perceived stress and depression after participating in the CVSM. VR has been observed to relieve anxiety and depression [45]. Some feasibility studies suggested that immersive virtual environment may promote the limited functioning of patients with cognitive impairment that affects communication, interaction, motivation, participation, and positive attitude toward others [46]. Therefore, the importance of virtual environment should be considered in cognitive training because the sense of existence in the virtual environment itself can enhance volitional motivation, allowing people to constantly deal with external stimuli and adapt to the changing environment in cognition [47]. There were few reports of mild dizziness during the early stages of the cognitive training period in this study, and simulator-related symptoms decreased as the number of cognitive training sessions increased. These results suggest that the CVSM is a feasible IVR system for older adults with MCI and MD. The low incidence of simulator disease in this study may be associated with the frequency and time of cognitive training programs. The neural mismatch model [48] indicates that unpleasant symptoms occur when the sensory information is inconsistent with the individual’s past experiences. Therefore, older adults are prone to discomfort when they enter some VR environments. The use of adaptation training programs that last for several days and the gradual increase of time spent using the simulator during a single training session can help prevent the discomfort associated with the simulator [49].

Some intervention studies have reported high dropout rates of older adults in cognitive training based on technical support [43,50]. Cognitive training based on Immersive Virtual can be regarded as an effective method to enhance user participation, which contributes to the positive results of intervention [51]. However, older adults are less exposed to modern digital technology in their daily life. Therefore, not all can understand and accept it. In order to overcome this limitation, this study carried out many interesting lectures related to VR and experience activities in nursing homes during the recruitment of research objects, so as to increase older adults' understanding of virtual technology and shorten the “digital gap” between older adults and technology. The results of the satisfaction survey suggest that the participants in this study were highly engaged during the intervention and had a positive attitude toward IVR-based cognitive training. The majority of the participants were attracted to this novel cognitive training method.

Limitations

This study is not without limitations. First, our pretest and posttests were identical. Older adults received a total score for each pretest but were not notified which answers they got correct on the pretest. Nevertheless, if they remembered the questions in the pretest, the identical pretests and posttests may have exaggerated the posttest scores. Second, the sample size was small and the study did not include a control group and a long-term follow-up analysis. However, according to the Virtual Reality Clinical Outcomes Research Experts framework [52], researchers are required to conduct early testing with a focus on feasibility, acceptability, tolerability, and initial clinical effectiveness before a standardized randomized controlled trial can be started. Therefore, this study is beneficial as it reduces the risk of conducting a randomized controlled trial with an intervention that has not undergone thorough testing. The effects and mechanisms of IVR-based cognitive training on the cognitive performance of patients with MCI and MD should be studied using high-quality randomized controlled trials.

Conclusion

In conclusion, this pilot study validated the feasibility and effectiveness of IVR-based cognitive training by using a novel CVSM system in older adults with MCI and MD. The results of this study support the use of IVR-based cognitive training in this patient population.

Acknowledgments

This study was funded by the Medical Innovation Project of Fujian Province, China (2020CXA002), the National Natural Science Foundation of China (82071222), and Joint Funds for the Innovation of Science and Technology, Fujian Province, China (2020Y9021). We thank all the participants and staff for their time and feedback. We also thank the Chinese virtual supermarket technical partners who allowed for the success of this study. We would like to thank Editage for English language editing. All authors take responsibility for the accuracy and completeness of the reported data.

Abbreviations

AVLT

Auditory Verbal Learning Test

CVSM

Chinese virtual supermarket

GDS

Geriatric Depression Scale

IVR

immersive virtual reality

MCI

mild cognitive impairment

MD

mild dementia

MoCA

Montreal Cognitive Assessment Scale

MMSE

Mini-Mental State Examination

PSS

Perceived Stress Scale

SDMT

Symbol Digit Modalities Test

STT

Shape Trail Test

VR

virtual reality

Footnotes

Conflicts of Interest: None declared.

References

  • 1.Ning Jizhe Main data of the seventh national census. National Bureau of Statistics of China. 2021. May 11, [2021-05-11]. http://www.stats.gov.cn/tjsj/zxfb/202105/t20210510_1817176.html .
  • 2.Zeng Y. Towards Deeper Research and Better Policy for Healthy Aging --Using the Unique Data of Chinese Longitudinal Healthy Longevity Survey. China Economic J. 2012;5(2-3):131–149. doi: 10.1080/17538963.2013.764677. http://europepmc.org/abstract/MED/24443653 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Jia L, Quan M, Fu Y, Zhao T, Li Y, Wei C, Tang Y, Qin Q, Wang F, Qiao Y, Shi S, Wang Y, Du Y, Zhang J, Zhang J, Luo B, Qu Q, Zhou C, Gauthier S, Jia J. Dementia in China: epidemiology, clinical management, and research advances. The Lancet Neurology. 2020 Jan;19(1):81–92. doi: 10.1016/s1474-4422(19)30290-x. [DOI] [PubMed] [Google Scholar]
  • 4.Academy of Cognitive Disorders of China (ACDC) Han Yuliang, Jia Jianjun, Li Xia, Lv Yang, Sun Xuan, Wang Y, Wang Z, Wang Zhiwen, Zhang Jintao, Zhou Y, Zhou Yuying. Expert Consensus on the Care and Management of Patients with Cognitive Impairment in China. Neurosci Bull. 2020 Mar;36(3):307–320. doi: 10.1007/s12264-019-00444-y. http://europepmc.org/abstract/MED/31792911 .10.1007/s12264-019-00444-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Jia J, Wei C, Chen S, Li F, Tang Y, Qin W, Zhao L, Jin H, Xu H, Wang F, Zhou A, Zuo X, Wu L, Han Y, Han Y, Huang L, Wang Q, Li D, Chu C, Shi L, Gong M, Du Y, Zhang J, Zhang J, Zhou C, Lv J, Lv Y, Xie H, Ji Y, Li F, Yu E, Luo B, Wang Y, Yang S, Qu Q, Guo Q, Liang F, Zhang J, Tan L, Shen L, Zhang K, Zhang J, Peng D, Tang M, Lv P, Fang B, Chu L, Jia L, Gauthier S. The cost of Alzheimer's disease in China and re-estimation of costs worldwide. Alzheimers Dement. 2018 Apr;14(4):483–491. doi: 10.1016/j.jalz.2017.12.006.S1552-5260(17)33877-3 [DOI] [PubMed] [Google Scholar]
  • 6.Couch E, Lawrence V, Co M, Prina M. Outcomes tested in non-pharmacological interventions in mild cognitive impairment and mild dementia: a scoping review. BMJ Open. 2020 Apr 20;10(4):e035980. doi: 10.1136/bmjopen-2019-035980. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=32317262 .bmjopen-2019-035980 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Bahar-Fuchs A, Martyr A, Goh A, Sabates Julieta, Clare Linda. Cognitive training for people with mild to moderate dementia. Cochrane Database Syst Rev. 2019 Mar 25;3:CD013069. doi: 10.1002/14651858.CD013069.pub2. http://europepmc.org/abstract/MED/30909318 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Szeto Jennifer Y Y, Lewis Simon J G. Current Treatment Options for Alzheimer's Disease and Parkinson's Disease Dementia. Curr Neuropharmacol. 2016;14(4):326–38. doi: 10.2174/1570159x14666151208112754. http://europepmc.org/abstract/MED/26644155 .CN-EPUB-72324 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Belleville S, Gilbert B, Fontaine F, Gagnon L, Ménard Edith, Gauthier S. Improvement of episodic memory in persons with mild cognitive impairment and healthy older adults: evidence from a cognitive intervention program. Dement Geriatr Cogn Disord. 2006;22(5-6):486–99. doi: 10.1159/000096316.96316 [DOI] [PubMed] [Google Scholar]
  • 10.Wenisch E, Cantegreil-Kallen I, De Rotrou J, Garrigue P, Moulin F, Batouche F, Richard A, De Sant'Anna Martha, Rigaud AS. Cognitive stimulation intervention for elders with mild cognitive impairment compared with normal aged subjects: preliminary results. Aging Clin Exp Res. 2007 Aug;19(4):316–22. doi: 10.1007/BF03324708.3843 [DOI] [PubMed] [Google Scholar]
  • 11.Liu Y, Tan W, Chen C, Liu C, Yang J, Zhang Y. A Review of the Application of Virtual Reality Technology in the Diagnosis and Treatment of Cognitive Impairment. Front Aging Neurosci. 2019;11:280. doi: 10.3389/fnagi.2019.00280. doi: 10.3389/fnagi.2019.00280. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Li B, He N, Qiao Y, Xu H, Lu Y, Cui P, Ling H, Yan F, Tang H, Chen S. Computerized cognitive training for Chinese mild cognitive impairment patients: A neuropsychological and fMRI study. Neuroimage Clin. 2019;22:101691. doi: 10.1016/j.nicl.2019.101691. https://linkinghub.elsevier.com/retrieve/pii/S2213-1582(19)30041-5 .S2213-1582(19)30041-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Gates Nicola J, Vernooij Robin Wm, Di Nisio Marcello, Karim Salman, March Evrim, Martínez Gabriel, Rutjes Anne Ws. Computerised cognitive training for preventing dementia in people with mild cognitive impairment. Cochrane Database Syst Rev. 2019 Mar 13;3:CD012279. doi: 10.1002/14651858.CD012279.pub2. http://europepmc.org/abstract/MED/30864747 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Mertz L. Virtual Reality Pioneer Tom Furness on the Past, Present, and Future of VR in Health Care. IEEE Pulse. 2019;10(3):9–11. doi: 10.1109/MPULS.2019.2911808. [DOI] [PubMed] [Google Scholar]
  • 15.García-Betances Rebeca I, Arredondo Waldmeyer MT, Fico G, Cabrera-Umpiérrez María Fernanda. A succinct overview of virtual reality technology use in Alzheimer's disease. Front Aging Neurosci. 2015;7:80. doi: 10.3389/fnagi.2015.00080. doi: 10.3389/fnagi.2015.00080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Rus-Calafell M, Garety P, Sason E, Craig TJK, Valmaggia LR. Virtual reality in the assessment and treatment of psychosis: a systematic review of its utility, acceptability and effectiveness. Psychol Med. 2018 Feb;48(3):362–391. doi: 10.1017/S0033291717001945.S0033291717001945 [DOI] [PubMed] [Google Scholar]
  • 17.Tuena C, Pedroli E, Trimarchi PD, Gallucci A, Chiappini M, Goulene K, Gaggioli A, Riva G, Lattanzio F, Giunco F, Stramba-Badiale M. Usability Issues of Clinical and Research Applications of Virtual Reality in Older People: A Systematic Review. Front Hum Neurosci. 2020;14:93. doi: 10.3389/fnhum.2020.00093. doi: 10.3389/fnhum.2020.00093. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Cho D, Lee S. Effects of virtual reality immersive training with computerized cognitive training on cognitive function and activities of daily living performance in patients with acute stage stroke: A preliminary randomized controlled trial. Medicine (Baltimore) 2019 Mar;98(11):e14752. doi: 10.1097/MD.0000000000014752. doi: 10.1097/MD.0000000000014752.00005792-201903150-00028 [DOI] [PMC free article] [PubMed] [Google Scholar] [Retracted]
  • 19.Nijman SA, Veling W, Greaves-Lord K, Vermeer RR, Vos M, Zandee CER, Zandstra DC, Geraets CNW, Pijnenborg GHM. Dynamic Interactive Social Cognition Training in Virtual Reality (DiSCoVR) for social cognition and social functioning in people with a psychotic disorder: study protocol for a multicenter randomized controlled trial. BMC Psychiatry. 2019 Sep 05;19(1):272. doi: 10.1186/s12888-019-2250-0. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-019-2250-0 .10.1186/s12888-019-2250-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Hwang J, Lee S. The effect of virtual reality program on the cognitive function and balance of the people with mild cognitive impairment. J Phys Ther Sci. 2017 Aug;29(8):1283–1286. doi: 10.1589/jpts.29.1283. http://europepmc.org/abstract/MED/28878448 .jpts-2017-034 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Burdea G, Polistico K, Krishnamoorthy A, House G, Rethage D, Hundal J, Damiani F, Pollack S. Feasibility study of the BrightBrainer™ integrative cognitive rehabilitation system for elderly with dementia. Disabil Rehabil Assist Technol. 2015;10(5):421–32. doi: 10.3109/17483107.2014.900575. http://europepmc.org/abstract/MED/24679074 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Sayma M, Tuijt R, Cooper C, Walters Kate. Are We There Yet? Immersive Virtual Reality to Improve Cognitive Function in Dementia and Mild Cognitive Impairment. Gerontologist. 2020 Sep 15;60(7):e502–e512. doi: 10.1093/geront/gnz132.5585509 [DOI] [PubMed] [Google Scholar]
  • 23.Eraslan Boz H, Limoncu H, Zygouris S, Tsolaki M, Giakoumis D, Votis K, Tzovaras D, Öztürk Vesile, Yener GG. A new tool to assess amnestic mild cognitive impairment in Turkish older adults: virtual supermarket (VSM) Neuropsychol Dev Cogn B Aging Neuropsychol Cogn. 2020 Sep;27(5):639–653. doi: 10.1080/13825585.2019.1663146. [DOI] [PubMed] [Google Scholar]
  • 24.Lecouvey G, Morand A, Gonneaud J, Piolino P, Orriols E, Pélerin Alice, Ferreira Da Silva L, de La Sayette V, Eustache F, Desgranges B. An Impairment of Prospective Memory in Mild Alzheimer's Disease: A Ride in a Virtual Town. Front Psychol. 2019;10:241. doi: 10.3389/fpsyg.2019.00241. doi: 10.3389/fpsyg.2019.00241. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Giovannetti T, Yamaguchi T, Roll E, Harada T, Rycroft SS, Divers R, Hulswit J, Tan CC, Matchanova A, Ham L, Hackett K, Mis R. The Virtual Kitchen Challenge: preliminary data from a novel virtual reality test of mild difficulties in everyday functioning. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn. 2019 Nov;26(6):823–841. doi: 10.1080/13825585.2018.1536774. [DOI] [PubMed] [Google Scholar]
  • 26.Man DWK, Chung JCC, Lee GYY. Evaluation of a virtual reality-based memory training programme for Hong Kong Chinese older adults with questionable dementia: a pilot study. Int J Geriatr Psychiatry. 2012 May;27(5):513–20. doi: 10.1002/gps.2746. [DOI] [PubMed] [Google Scholar]
  • 27.Mrakic-Sposta S, Di Santo SG, Franchini F, Arlati S, Zangiacomi A, Greci L, Moretti S, Jesuthasan N, Marzorati M, Rizzo G, Sacco M, Vezzoli A. Effects of Combined Physical and Cognitive Virtual Reality-Based Training on Cognitive Impairment and Oxidative Stress in MCI Patients: A Pilot Study. Front Aging Neurosci. 2018;10:282. doi: 10.3389/fnagi.2018.00282. doi: 10.3389/fnagi.2018.00282. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Moyle W, Jones C, Dwan T, Petrovich Tanya. Effectiveness of a Virtual Reality Forest on People With Dementia: A Mixed Methods Pilot Study. Gerontologist. 2018 May 08;58(3):478–487. doi: 10.1093/geront/gnw270.3072156 [DOI] [PubMed] [Google Scholar]
  • 29.Brown JA. An Exploration of Virtual Reality Use and Application Among Older Adult Populations. Gerontol Geriatr Med. 2019;5:2333721419885287. doi: 10.1177/2333721419885287. https://journals.sagepub.com/doi/10.1177/2333721419885287?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed .10.1177_2333721419885287 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Park MJ, Kim DJ, Lee U, Na EJ, Jeon HJ. A Literature Overview of Virtual Reality (VR) in Treatment of Psychiatric Disorders: Recent Advances and Limitations. Front Psychiatry. 2019;10:505. doi: 10.3389/fpsyt.2019.00505. doi: 10.3389/fpsyt.2019.00505. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.D'Cunha NM, Nguyen D, Naumovski N, McKune AJ, Kellett J, Georgousopoulou EN, Frost J, Isbel S. A Mini-Review of Virtual Reality-Based Interventions to Promote Well-Being for People Living with Dementia and Mild Cognitive Impairment. Gerontology. 2019;65(4):430–440. doi: 10.1159/000500040. https://www.karger.com?DOI=10.1159/000500040 .000500040 [DOI] [PubMed] [Google Scholar]
  • 32.Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004 Sep;256(3):183–94. doi: 10.1111/j.1365-2796.2004.01388.x. https://onlinelibrary.wiley.com/resolve/openurl?genre=article&sid=nlm:pubmed&issn=0954-6820&date=2004&volume=256&issue=3&spage=183 .JIM1388 [DOI] [PubMed] [Google Scholar]
  • 33.The ICD-10 classification of mental and behavioral disorders: clinical descriptions and diagnostic guidelines. World Health Organization. [2022-02-15]. https://www.who.int/substance_abuse/terminology/ICD10ClinicalDiagnosis.pdf .
  • 34.Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology. 1993 Jul;3(3):203–220. doi: 10.1207/s15327108ijap0303_3. [DOI] [Google Scholar]
  • 35.Trivedi D. Cochrane Review Summary: Mini-Mental State Examination (MMSE) for the detection of dementia in clinically unevaluated people aged 65 and over in community and primary care populations. Prim Health Care Res Dev. 2017 Nov;18(6):527–528. doi: 10.1017/S1463423617000202.S1463423617000202 [DOI] [PubMed] [Google Scholar]
  • 36.Nasreddine Ziad S, Phillips Natalie A, Bédirian Valérie, Charbonneau Simon, Whitehead Victor, Collin Isabelle, Cummings Jeffrey L, Chertkow Howard. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005 Apr;53(4):695–9. doi: 10.1111/j.1532-5415.2005.53221.x.JGS53221 [DOI] [PubMed] [Google Scholar]
  • 37.Putcha D, Brickhouse M, Wolk DA, Dickerson BC, Alzheimer's Disease Neuroimaging Initiative Fractionating the Rey Auditory Verbal Learning Test: Distinct roles of large-scale cortical networks in prodromal Alzheimer's disease. Neuropsychologia. 2019 Jun;129:83–92. doi: 10.1016/j.neuropsychologia.2019.03.015. http://europepmc.org/abstract/MED/30930301 .S0028-3932(18)30637-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Zhao Q, Guo Q, Li F, Zhou Y, Wang B, Hong Z. The Shape Trail Test: application of a new variant of the Trail making test. PLoS One. 2013;8(2):e57333. doi: 10.1371/journal.pone.0057333. https://dx.plos.org/10.1371/journal.pone.0057333 .PONE-D-12-29095 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Ryan J, Woods RL, Britt CJ, Murray AM, Shah RC, Reid CM, Wolfe R, Nelson MR, Orchard SG, Lockery JE, Trevaks RE, Storey E. Normative Data for the Symbol Digit Modalities Test in Older White Australians and Americans, African-Americans, and Hispanic/Latinos. ADR. 2020 Aug 04;4(1):313–323. doi: 10.3233/adr-200194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Smarr KL, Keefer AL. Measures of Depression and Depressive Symptoms. Arthritis Care Res (Hoboken) 2020 Oct;72 Suppl 10:608–629. doi: 10.1002/acr.24191. [DOI] [PubMed] [Google Scholar]
  • 41.Deeken F, Häusler A, Nordheim J, Rapp M, Knoll N, Rieckmann N. Psychometric properties of the Perceived Stress Scale in a sample of German dementia patients and their caregivers. Int. Psychogeriatr. 2017 Jul 24;30(1):39–47. doi: 10.1017/s1041610217001387. [DOI] [PubMed] [Google Scholar]
  • 42.Kim O, Pang Y, Kim J. The effectiveness of virtual reality for people with mild cognitive impairment or dementia: a meta-analysis. BMC Psychiatry. 2019 Jul 12;19(1):219. doi: 10.1186/s12888-019-2180-x. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-019-2180-x .10.1186/s12888-019-2180-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Zając-Lamparska L, Wiłkość-Dębczyńska M, Wojciechowski A, Podhorecka M, Polak-Szabela A, Warchoł ?, Kędziora-Kornatowska K, Araszkiewicz A, Izdebski P. Effects of virtual reality-based cognitive training in older adults living without and with mild dementia: a pretest-posttest design pilot study. BMC Res Notes. 2019 Nov 27;12(1):776. doi: 10.1186/s13104-019-4810-2. https://bmcresnotes.biomedcentral.com/articles/10.1186/s13104-019-4810-2 .10.1186/s13104-019-4810-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Otsuka T, Tanemura R, Noda K, Nagao T, Sakai H, Luo Z. Development of Computer-Aided Cognitive Training Program for Elderly and Its Effectiveness through a 6 Months Group Intervention Study. Curr Alzheimer Res. 2015;12(6):553–62. doi: 10.2174/1567205012666150530202335.CAR-EPUB-67749 [DOI] [PubMed] [Google Scholar]
  • 45.Qian J, McDonough DZ, Gao Zan. The Effectiveness of Virtual Reality Exercise on Individual's Physiological, Psychological and Rehabilitative Outcomes: A Systematic Review. Int J Environ Res Public Health. 2020 Jun 10;17(11):1661–7827. doi: 10.3390/ijerph17114133. https://www.mdpi.com/resolver?pii=ijerph17114133 .ijerph17114133 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Garcia L, Kartolo A, Méthot-Curtis E. Virtual Reality in Psychological, Medical and Pedagogical Applications. Rijeka: Intech; 2012. [Google Scholar]
  • 47.Riva G, Waterworth JA, Waterworth EL, Mantovani F. From intention to action: The role of presence. New Ideas in Psychology. 2011 Jan;29(1):24–37. doi: 10.1016/j.newideapsych.2009.11.002. [DOI] [Google Scholar]
  • 48.Reason JT. Motion sickness adaptation: a neural mismatch model. J R Soc Med. 1978 Nov;71(11):819–29. doi: 10.1177/014107687807101109. http://europepmc.org/abstract/MED/731645 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Dużmańska N, Strojny P, Strojny A. Can Simulator Sickness Be Avoided? A Review on Temporal Aspects of Simulator Sickness. Front Psychol. 2018;9:2132. doi: 10.3389/fpsyg.2018.02132. doi: 10.3389/fpsyg.2018.02132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Liu Q, Wang Y, Yao MZ, Tang Q, Yang Y. The Effects of Viewing an Uplifting 360-Degree Video on Emotional Well-Being Among Elderly Adults and College Students Under Immersive Virtual Reality and Smartphone Conditions. Cyberpsychol Behav Soc Netw. 2020 Mar;23(3):157–164. doi: 10.1089/cyber.2019.0273. [DOI] [PubMed] [Google Scholar]
  • 51.Syed-Abdul S, Malwade S, Nursetyo AA, Sood M, Bhatia M, Barsasella D, Liu MF, Chang C, Srinivasan K, Li Yu-Chuan Jack. Virtual reality among the elderly: a usefulness and acceptance study from Taiwan. BMC Geriatr. 2019 Aug 19;19(1):223. doi: 10.1186/s12877-019-1218-8. https://bmcgeriatr.biomedcentral.com/articles/10.1186/s12877-019-1218-8 .10.1186/s12877-019-1218-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Birckhead B, Khalil C, Liu X, Conovitz S, Rizzo A, Danovitch I, Bullock K, Spiegel B. Recommendations for Methodology of Virtual Reality Clinical Trials in Health Care by an International Working Group: Iterative Study. JMIR Ment Health. 2019 Jan 31;6(1):e11973. doi: 10.2196/11973. https://mental.jmir.org/2019/1/e11973/ v6i1e11973 [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from JMIR Serious Games are provided here courtesy of JMIR Publications Inc.

RESOURCES