Is it possible to use highly realistic virtual reality in the elderly? A feasibility study with image-based rendering

Abstract

Background

Virtual reality (VR) opens up a vast number of possibilities in many domains of therapy. The primary objective of the present study was to evaluate the acceptability for elderly subjects of a VR experience using the image-based rendering virtual environment (IBVE) approach and secondly to test the hypothesis that visual cues using VR may enhance the generation of autobiographical memories.

Methods

Eighteen healthy volunteers (mean age 68.2 years) presenting memory complaints with a Mini-Mental State Examination score higher than 27 and no history of neuropsychiatric disease were included. Participants were asked to perform an autobiographical fluency task in four conditions. The first condition was a baseline grey screen, the second was a photograph of a well-known location in the participant’s home city (FamPhoto), and the last two conditions displayed VR, ie, a familiar image-based virtual environment (FamIBVE) consisting of an image-based representation of a known landmark square in the center of the city of experimentation (Nice) and an unknown image-based virtual environment (UnknoIBVE), which was captured in a public housing neighborhood containing unrecognizable building fronts. After each of the four experimental conditions, participants filled in self-report questionnaires to assess the task acceptability (levels of emotion, motivation, security, fatigue, and familiarity). CyberSickness and Presence questionnaires were also assessed after the two VR conditions. Autobiographical memory was assessed using a verbal fluency task and quality of the recollection was assessed using the “remember/know” procedure.

Results

All subjects completed the experiment. Sense of security and fatigue were not significantly different between the conditions with and without VR. The FamPhoto condition yielded a higher emotion score than the other conditions (P<0.05). The CyberSickness questionnaire showed that participants did not experience sickness during the experiment across the VR conditions. VR stimulates autobiographical memory, as demonstrated by the increased total number of responses on the autobiographical fluency task and the increased number of conscious recollections of memories for familiar versus unknown scenes (P<0.01).

Conclusion

The study indicates that VR using the FamIBVE system is well tolerated by the elderly. VR can also stimulate recollections of autobiographical memory and convey familiarity of a given scene, which is an essential requirement for use of VR during reminiscence therapy.

Video abstract

Download video file ^{(89.4MB, avi)}

Introduction

Information and communication technologies (ICT) have proved to be useful in assessing and evaluating patients’ functional impairments, as well as in helping and supporting patients in everyday activities.^1,2 Furthermore, ICT can play a key role in the stimulation and rehabilitation of disabled patients. For instance, Smartphones³ or SenseCams allowing capture of the subject’s personal experience have been shown to help retrieval in patients with memory difficulties.⁴ The use of ICT in cognitive disorders is expected, since it is recognized that these disorders are frequent in the elderly, play a major part in individual adaptation to the environment, and contribute to a poorer quality of life.

Virtual reality (VR), defined as a computer-simulated environment that can provide the sensation of physical presence in places representing real or imagined worlds, opens up a vast number of additional possibilities in many domains of therapy, including rehabilitation such as post-traumatic stress disorder⁵ and cognition^6,7 and in many domains of therapy.^6–10 Despite the fact that computer-based neuropsychological assessment is extensively used in elderly population,¹¹ surprisingly few studies of VR have included old subjects.

Some factors associated with old age have been assumed to limit the use of VR in this population, such as coping with computerized or virtual environments, engagement in rigid procedures, and age-related or pathological cognitive impairment. These factors may contribute to higher interindividual differences in ICT tasks than with traditional (verbal, page-pencil) assessments.¹² It has also been suggested that psychological factors would limit the use of VR in a population that is often seen as less involved in new technologies. Even if a cohort effect is frequently found with the use of many technical appliances, old people may engage in new tasks after a period of gradual familiarization, even when affected by cognitive disorders.¹³ Age-related physical limitations, such as retardation or sickness, might also limit the use of very reactive virtual environments. In contrast, most procedures may be adapted to compensate for individual slowness, making it possible to measure and take into account motion sickness before interpretation of the performance. This beneficial effect of VR with regard to compensating for physical deficit has been particularly demonstrated in severe clinical situations, such as stroke rehabilitation.^14,15 Therefore, there is no rationale to keep old people away from new technologies, but it is necessary to confirm that they can accept a virtual environment and interact cognitively with it.

Among the cognitive functions, autobiographical memory has not been tested with old-age people in a virtual environment. Stimulation of autobiographical memory might be used for reminiscence therapy, which has demonstrated its efficacy in the non-pharmacological treatment of dementia.

Autobiographical memory is one of the essential components of human memory. It is defined as mental representations of events from one’s past and semantic information about the self.¹⁶ The retrieval of autobiographical memories is a complex process in which mental traces of past events and related semantic knowledge are shaped into mental representations in light of current goals and concerns. These mental representations can differ across many qualities, including their content, imagery, degree of consciousness, and emotional intensity. Autobiographical memories contribute to the development of a coherent sense of ourselves, our emotions, and our future plans,¹⁷ and play a major role in motivating behavior and in daily life. Episodic autobiographical memory, in particular the richness of detail, is impaired early in the course of Alzheimer’s disease and perhaps even in the preclinical phase, while semantic memories are preserved until the more advanced stages, indicating a dissociation between both memory systems.^18,19

Autobiographical memories are one of the processes targeted by reminiscence therapy. Reminiscence therapy consists of discussion of past activities, events, and experiences with another person or group of people, usually with the aid of tangible prompts, such as photographs, household and other familiar items from the past, and music or archive sound recordings. Reminiscence therapy is one of the most popular psychosocial interventions in dementia care, and is highly rated by staff and participants. A review of the literature shows some improvement of functional ability in people with dementia but the number of well controlled studies in the field is limited.²⁰

One of the most effective ways in which reminiscence therapy can be used to trigger autobiographical memories is the use of words and visual or other sensory cues. VR may be a promising tool in reminiscence therapy because of its visual realism and high degree of immersion. However, to our knowledge, no specific study has been conducted so far to test the potential value of VR specifically for reminiscence therapy. One possible reason for this lack of research is that it is very difficult to create a virtual environment that is sufficiently realistic and related to the subject’s history, for example, highly realistic three-dimensional places that are familiar to users. Manually creating such models specifically for the individual user is expensive and time-consuming, and often does not result in virtual images of sufficient realism. Another possible reason is that the more realistic the VR, the greater the chances of developing side effects, the most frequent of which is cybersickness,²¹ a form of motion sickness with symptoms reported to include nausea, vomiting, eyestrain, disorientation, ataxia, vertigo, and drowsiness.

This pilot study evaluated the acceptability to elderly subjects of a VR experience using the image-based rendering virtual environment (IBVE) approach.²² In order to test VR, we used an experiment inspired by reminiscence therapy which is known to help in the generation of autobiographical memories.

Materials and methods

Participants and clinical assessment

Participants were 18 healthy older volunteers recruited by the Nice Memory Center. At the time of inclusion, subjects underwent a clinical examination including screening for motor and visual deficiencies. Subjects were included if they had memory problems and a Mini-Mental State Examination score²³ higher than 27. Subjects were not included if they had a clinical history of neuropsychiatric disease, dementia, or vision difficulties. Ethical approval was obtained from an Individuals Protection Committee. Initial assessment included the Mini-Mental State Examination, Short Cognitive Battery,²⁴ Frontal Assessment Battery,²⁵ Apathy Inventory,²⁶ and Unified Parkinson’s Disease Rating Scale.²⁷

Experimental conditions and systems technology

Participants were presented with four experimental conditions (Figure 1). Two conditions did not use VR (gray/FamPhoto) and the two other conditions included VR, one being familiar (FamIBVE) and one being unknown (UnknoIBVE). The subjects spent a maximum of 15 minutes in the VR conditions (ie, 7 minutes in each VR condition). After each condition, participants were asked whether they recognized the environment that had been presented. The VR hardware setup was located in the INRIA (Institut National de Recherche en Informatique et en Automatique) Sophia Antipolis center (Figure 2).

Figure 1.

Four visual conditions used during the experiment and presented to participants in the following order.

Notes: (A) Baseline condition (gray, a medium gray blank screen). In accordance with the classical use of pictures in reminiscence therapy, participants were presented with (B) a photograph of a well-known location in the participant’s home city in Nice (FamPhoto). The two conditions of virtual reality are presented in a random order; ie, (C) a familiar image-based virtual environment (FamIBVE) consisting of an image-based representation of a known landmark in the center of the city of experimentation (Nice), and (D) an unknown image-based virtual environment (UnknoIBVE) captured in a public housing neighborhood, containing unrecognizable building fronts.

Abbreviation: IBVE, image-based virtual environment.

Figure 2.

Hardware setup used for the experiments.

Notes: To avoid any risk of unsteadiness or falling, each participant sat on a chair installed in front of the display screen, at a 1 meter distance. The IBVEs were captured by taking 90 photographs for each scene (FamIBVE and UnknoIBVE, respectively). In order to allow in-place navigation and interactive manipulation of objects, the photographs used for the IBVEs have a narrow field of view to provide a scale 1 immersive experience on the large screen. A small stool is placed next to the chair for the experimenter. The projectors use passive Infitec stereo via glasses, which are tracked using the ART tracking system (http://www.ar-tracking.com). In addition, we use the ART wireless finger-tracking of the orientation of the hand and the position of the fingers. We use the three-finger (thumb-index-middle finger) version. In order to increase the sensation of immersion in the virtual environment and allow participants to become familiar with finger tracking, we start the two IBVE sessions with an interactive manipulation task. This involves manipulating three-dimensional synthetic objects in the IBVEs. Specifically a plate with two dishes (one next to the other) is presented to the participant. The right dish contains three apples. The apples are dynamic rigid objects following gravity rules, controlled by a physics engine. Finally, to enrich the sensation of immersion in the virtual reality environment, we added ambient spatialized three-dimensional sound rendering (eg, street, car, and ambient noise, crowds walking and talking). Spatialized sound feedback related to the dynamic virtual objects is provided when the apples are removed or placed on the dishes.

Abbreviation: IBVE, image-based virtual environment; FamIBVE, familiar image-based virtual environment; UnknoIBVE, unknown image-based virtual environment.

One of the technical challenges of this type of project is the development and use of an image-based rendering system²² that can easily capture and display a familiar and personalized environment with photographs as a starting point. The system also provides support for near-normal interaction with objects in the environment with finger tracking. At the time of the experiment, development of the image-based rendering algorithm we used was limited to a single screen, so we used only the front wall of a BARCO iSpace for our experiments. The size of the screen is 320 cm ×240 cm with a display resolution of 1,600×1,200 pixels. The projectors use passive Infitec stereo via glasses that are tracked using the ART tracking system (Advanced Realtime Tracking GmbH, Weilheim in Oberbayern, Germany).

Questionnaires, other dependent measures, data analysis

After exposure to each of the four experimental conditions, participants completed up to five self-report questionnaires assessing acceptability (levels of emotion, motivation, security, fatigue, and familiarity). These questionnaires were devised using classical Likert scale methodology (a horizontal line, on which a subject indicates his or her response by checking tick marks between 0 and 10). Cyber-Sickness and Presence questionnaires were only completed after the two VR conditions, and were tested respectively with the 22-item Cybersickness questionnaire,²⁸ and the general and spatial presence questionnaire.²⁹ The questionnaire scores are explained in the notes for Table 1.

Table 1.

Acceptability and autobiographical memory results in the four conditions

	Gray	FamPhoto	UnknoIBVE	FamIBVE
Emotion	4.25 (3.40)	7.27 (2.08)*	3.60 (2.99)	4.93 (2.99)
Motivation	5.81 (3.80)	8.64 (0.96)**	6.44 (3.43)**	7.71 (2.84)**
Security	8.25 (2.70)	9.39 (0.44)	7.50 (3.17)	8.42 (2.61)
Fatigue	1.01 (1.54)	0.58 (0.27)	1.08 (1.37)	0.72 (0.49)
Anxiety		7.05 (3.32)	7.23 (2.86)	7.24 (3.22)
Cybersickness			0.46 (0.1)	1.42 (0.3)
Presence			39.7 (22)	42.8 (22)
AF total number of responses	6.4 (2.6)	10.4 (3.6)***	7.3 (3.28)	10.8 (4.2)***
Remember	1 (1.06)	5 (3.4)****	2.6 (1.8)	5.4 (2.6)****
Know	1 (0.8)	0.8 (1.5)	0.4 (0.68)	0.9 (0.9)

Notes: Gray indicates a medium gray blank screen; FamPhoto indicates a classical two-dimensional photograph of a familiar city; UnknoIBVE indicates an unknown image-based virtual environment. FamIBVE indicates a familiar image-based virtual environment. Emotion, motivation, security, fatigue, and anxiety scores are expressed as the mean ± SD on a scale from 0 (no emotion, motivation, security, fatigue) to 10 (high emotion, motivation, security, fatigue). Anxiety was scored from 0 (very anxious) to 10 (no anxiety) and was not scored during the gray condition. Cybersickness questionnaire scores are expressed as the mean ± SD on a scale from 0 (no cybersickness) to 88 (high cybersickness). Twenty-two items (each rating from 0 to 4). Presence questionnaire scores are expressed as the mean ± SD on a scale from 0 (no presence) to 84 (high presence). Fourteen items (each rating from 0 to 6). AF responses are expressed as the mean ± SD representing the number of remember responses. Remember responses and know responses (according to the remember/know procedure) are expressed as the mean ± SD.

^*P<0.05 versus other conditions

^**P<0.05 versus gray (Wilcoxon signed rank test., two-tailed)

^***P<0.05 FamIBVE or photograph versus gray or UnKnoIBVE

^****P<0.001 FamIBVE or photograph versus gray or UnKnoIBVE.

Abbreviations: AF, autobiographical memories fluency task; SD, standard deviation.

In order to assess autobiographical memories, we adapted the autobiographical fluency task³⁰ to the context of the experiment. Subjects were placed in an experimental room, and while presented with visual material, they were asked to generate as many autobiographical memories as possible related to the presented environment in 2 minutes using short sentences. At the end of the session, the investigator assessed the quality of the recollection using the “remember/know” procedure.³¹ Each subject was asked to indicate for each of his/her recollections if he/she remembered it (conscious recollection of many vivid contextual details, such as “when” and “how” the information was learned) or if he/she knew it (the sensation that the event/environment had been seen before, but not being able to identify the reason why). The subject’s verbal outputs were directly recorded during the experiment and analyzed after the session by a speech therapist blinded to the research protocol. The score derived included the number of memories and number of remember and know recollections.

The statistical analysis was carried out using Statistical Package for the Social Sciences version 11.0 software (SPSS Inc., Chicago, IL, USA) with a significance level of 0.05. Values are expressed as the mean ± standard deviation. Mean comparisons between the conditions were performed using the Wilcoxon non-parametric test.

Results

Eighteen healthy subjects (eleven men, seven women; mean age 68.2±7.8 years) were included (Table 2). Cognitive performance was in the normal range after controlling for age and educational level, and participants had no symptomatology on the Apathy Inventory or Unified Parkinson’s Disease Rating Scale. All subjects completed the experiment. Five participants were excluded from the autobiographical memory analysis because they were not living in Nice or surrounding area, so did not recognize the familiar town pictures.

Table 2.

Participants’ demographic and clinical characteristics

Subjects	n=18, mean (SD) [min, max]
Age, years	68.2 (7.8)
Sex	7 female, 11 male
MMSE	29.2 (0.8) [28, 30]
Clock Drawing*	6.65 (0.5) [6, 7]
Five Words Recall Test*	10 (0) [10, 10]
Verbal Fluency category*	21.88 (7.0) [6, 33]
Frontal Assessment Battery	16.76 (1.8) [12, 18]
Apathy Inventory	0 (0) [0, 0]
UPDRS	0 (0) [0, 0]

Notes: Clock Drawing (maximum score 7); Five Words Recall Test (maximum score 7); Verbal Fluency animals in 1 minute (cutoff score for pathological performances 14); Frontal Assessment Battery (maximum score 18); Apathy Inventory (maximum score 12);

^*Clock Drawing, Five Words Recall Test, and Verbal Fluency belong to the Short Cognitive Battery.

Abbreviations: max, maximum; min, minimum; MMSE, Mini Mental Score Evaluation; SD, standard deviation; UPDRS, Unified Parkinson’s Disease Rating Scale.

Table 1 shows the acceptability and autobiographical memory results in the four conditions. Sense of security and fatigue were not significantly different between the four conditions. Motivation during the baseline (gray) condition was significantly lower than in the other conditions (P<0.05), motivation not different between the VR conditions and Fam-Photo condition. The FamPhoto condition yielded a higher emotion score than the other conditions (P<0.05).

For the two VR conditions, data from the CyberSickness questionnaire indicated that participants did not experience important sickness (0, no cybersickness; 6, significant cyber-sickness) with UnknoIBVE (mean 0.46±0.1) or FamIBVE (mean 1.42±0.3). Scores on the Presence questionnaire were in the medium range (on a scale from 0 to 84). There were no significant differences between the two conditions for either the CyberSickness (P=0.11) or Presence (P=0.24) questionnaire.

Concerning the autobiographical fluency task, the total number of responses in the FamIBVE and FamPhoto conditions were significantly (P<0.05) higher than those in the gray and UnknoIBVE conditions. The number of “remember” responses was also significantly higher in the FamIBVE and FamPhoto conditions (P<0.01) than those observed in the gray and UnknoIBVE conditions. No other comparisons reached statistical significance.

Discussion

This pilot study indicates that it is possible to use VR environments with elderly subjects. As reported in the questionnaires, the subjects felt that virtual environments were secure and did not report any significant fatigue. None of the subjects dropped out during the study. During the conditions employing VR, the degree of cybersickness, which is one classical weakness of the technique, was very low. This result was probably reinforced by the physical apparatus used (a chair) in the immersive room, allowing the participants to be seated beside a professional during the experience.

We also wanted to test the hypothesis that visual cues provided by VR can enhance the production of autobiographical memories, and more specifically that cues involving an environment related to the subject’s personal biography might generate better memory performance than a neutral environment.

Our results indicate that VR using IBVE stimulates autobiographical memory, as demonstrated by the increased total number of responses, as well as by the increased number of conscious recollections of memories for familiar versus unknown scenes. From a technical standpoint, the higher number of memories generated with FamIBVE than with UnknoIBVE is an indication that the IBVE system is able to convey familiarity in the immersive setting, but only when the environment is related to the individual’s personal history. However, performance observed with the VR environment was not superior to performance observed with a static photograph. One possible explanation relates to technical limitations arising from the use of a single screen (Figure 2). This restriction conducted to use a limited narrow field of view to allow a presentation at scale 1 of the environment on the screen. In contrast with the narrow field offered in the VR condition, the field of view of the photograph was much larger, offering the potential to elicit more emotion and memories.

Future studies should investigate whether this interpretation is correct. For instance, we expect that use of IBVE will increase when multiple screens become available, since the field of view will be much larger. This technical improvement will allow capturing in the VR environment, photographs with a larger field of view and offer a higher immersion experience.

This preliminary study has several limitations. We did not include a control group tested in a classical environment without any technological interface. Due to the nonavailability of the experimental room in a clinical setting, the number of participants was small. The autobiographical memory results suggest that it may be interesting to use this VR system in a therapeutic context, for instance in patients with memory impairment, such as Alzheimer’s disease. However, we only included healthy elderly volunteers in the present study.

In conclusion, from a clinical perspective, the results of this study underline two main points. First, IBVE is suitable for use in the elderly. Second, use of VR can stimulate recollections of autobiographical memory. This pilot study will help to develop another experiment using virtual environment with patients in a real-care setting in order to confirm its promise for clinical care.