Skip to main content
NCBI home page
Human Brain Mapping logoLink to Human Brain Mapping
. 2009 Dec 4;31(7):1065–1075. doi: 10.1002/hbm.20917

Impact of the virtual reality on the neural representation of an environment

Emmanuel Mellet 1,✉, Laetitia Laou 1, Laurent Petit 1, Laure Zago 1, Bernard Mazoyer 1,2,3, Nathalie Tzourio‐Mazoyer 1
PMCID: PMC6870852  PMID: 19967769

Abstract

Despite the increasing use of virtual reality, the impact on cerebral representation of topographical knowledge of learning by virtual reality rather than by actual locomotion has never been investigated. To tackle this challenging issue, we conducted an experiment wherein participants learned an immersive virtual environment using a joystick. The following day, participants' brain activity was monitored by functional magnetic resonance imaging while they mentally estimated distances in this environment. Results were compared with that of participants performing the same task but having learned the real version of the environment by actual walking. We detected a large set of areas shared by both groups including the parieto‐frontal areas and the parahippocampal gyrus. More importantly, although participants of both groups performed the same mental task and exhibited similar behavioral performances, they differed at the brain activity level. Unlike real learners, virtual learners activated a left‐lateralized network associated with tool manipulation and action semantics. This demonstrated that a neural fingerprint distinguishing virtual from real learning persists when subjects use a mental representation of the learnt environment with equivalent performances. Hum Brain Mapp, 2010. © 2009 Wiley‐Liss, Inc.

Keywords: spatial cognition, topographic representation, virtual reality, learning, fMRI

INTRODUCTION

In everyday life, most topographical knowledge is acquired through navigation within the environment (“route” perspective), although maps (“survey” perspective) or descriptive texts (either route or survey perspective) offer alternative means to build internal topographic representations. We have shown that differences in the way in which topographical knowledge is encoded lead to differences in the mobilization of neural networks during mental exploration of a learned topography. For example, mental scanning of a map learned from verbal description activated language areas, while mental scanning of the same map learned visually deactivated these same areas [Mellet et al., 2002]. These results suggested that an effect of the type of learning remains present in the cortical representation of the learned environment. Would this effect still be present when the only difference between the two modalities is the virtual or real nature of the environment learned? Even though the use of realistic simulation devices grows in ever more various domains, including neuroimaging studies, the question of whether virtual environments (VE) are represented differently from real environment (RE) within the brain is still unknown. In contrast to what has been observed with learning by visual map or descriptive text, spatial knowledge acquired by moving in VE (in a route perspective) is broadly similar to that gained from walking within a RE, at least when the virtual environment used is highly realistic (immersive VE) and has a simple spatial configuration [Lessels and Ruddle, 2005; Richardson et al., 1999; Ruddle et al., 1997].

However, even if immersive VE provides a realistic display, a crucial difference remains compared with the real world: exploring and learning a VE usually involves simulated locomotion through a joystick. It thus requires the conversion of visual information arising from the visual display into motor manipulation of the joystick to control one's trajectory within the virtual space. If, as suggested above, the type of learning shapes the cortical representation of an environment, then the use of a tool dedicated to navigation should have consequences at the cortical level. In particular, the process related to the complex use of a tool for learning should leave a cortical effect. We designed an experiment wherein participants freely navigated using a joystick within an immersive virtual indoor environment containing 10 landmarks. A control group explored the real version of a strictly identical environment by actual walking. The day after the learning phase, all subjects were scanned with fMRI while performing a topographical memory task where they had to mentally compare bird‐flight distances between pairs of landmarks in the previously learned environment. We postulated that differences between virtual and real learners should reflect the fact that learning a virtual environment is based on praxis while learning a real environment involved locomotion. In this framework, we expected more prevalent activations in the left hemisphere, specialized for complex tool use, thus reflecting the use of a joystick the day before, during learning [Johnson‐Frey, 2004]. More specifically, we hypothesized that retrieval and mental scanning of a representation of an environment built from virtual exploration through a tool should involve left lateralized areas implied in visuomotor associative learning and visumotor transformation, such as the inferior parietal cortex [Fogassi and Luppino, 2005; Naito and Ehrsson, 2006], and inferior frontal cortex [Binkofski and Buccino, 2004; Goldenberg et al., 2007]. We also paid a particular attention to the activity in the left posterior middle temporal cortex involved in functional knowledge about tools [Chao et al., 1999; Johnson‐Frey, 2004; Kellenbach et al., 2003; Noppeney et al., 2005].

MATERIALS AND METHODS

Participants

Thirty healthy, right‐handed men (age, 19–30 years) were included in this study. All participants were free of neurological disease and injury and had no abnormality on T1‐weighted magnetic resonance imaging (MRI). To increase the homogeneity of the sample regarding spatial imagery abilities, participants' scores to the Mental rotations test (MRT) had to be larger than 10 [Vandenberg and Kuse, 1978]. Scores of the 30 participants were: Mean; 15.6, SD: 2.8. The local Ethics Committee approved this study and written informed consent was obtained from each participant. The subject sample was split into two learning groups: virtual learners (17 men), who learned an exact virtual replication of a building from virtual navigation, and real learners (13 men), who learned the spatial layout of the actual building from real navigation. None of the participants had seen the environment to be learnt before the experiment.

Materials

Environments

The virtual environment, developed by the System and Technology Integration Laboratory (CEA, Fontenay‐aux‐Roses), was an immersive virtual replication of a part of our laboratory. This replication had exactly the same topography as the real building (two parallel corridors of 25 m long linked by a transversal corridors of 10 m, see Fig. 1C) and contained ten landmarks set on the floor (Fig. 1A). The real environment, included the same ten landmarks at the same locations. (Fig. 1B)

Figure 1.

Figure 1

Open in a new tab

Environments and distance comparison task. (A) Part of the real environment. (B) Part of the virtual environment. (C) Aerial view of the environment (never seen by participants) containing 10 landmarks represented by red dots. Dashed lines represent an example of a pair of bird‐flight distances to be compared by participants (Su, suitcase; D, dustbin; St, stool; W, wheelchair; P, plant; E, extinguisher; C, chair; B, box; F, fan; H, hall stand). The green dot corresponds to the viewpoint A and B. (D) Setting up of the virtual learning.

This immersive virtual environment was displayed on a Pentium 4 bi‐processor using Virtools 3.0 software on a Barco I‐Space 2 stereoscopic display. The participant stood 1.5 m in front of a 2 × 2.5 m screen and wore CrystalEyes stereo shutter glasses that enabled the participant to perceive the virtual environment as an immersive three‐dimensional scene. The participant controlled his navigation within the virtual environment by manipulating an Intersense IS900 joystick with his right hand. He could go forwards, go backwards, turn left, turn right, or stop. During the exploration, the navigation speed could not exceed 1.2 m/s in order to match actual walking speed.

We chose a rather simple environment in order to minimize potential behavioral differences during learning the environment and while performing the mental task. Any cerebral difference could thus be attributed to the type of learning.

Procedures

Learning phase (out of the MR scanner)

The day before fMRI scanning, participants had to explore and learn the environment either from navigation through the virtual replication of the laboratory (virtual learners) or from actual navigation (real learners). Each participant of both groups was instructed to navigate alone within the learning environment (real or virtual) and to memorize the topography of the environment and the location of 10 landmarks as accurately as possible and with no time limit. Both groups of participants started the exploration from the same starting point and encountered the landmarks successively in the same order. This learning phase was followed by a cued recall task during which participants had to recall all the landmark locations without mistake. In case of any mistake, a second exploration was performed by the subject and so forth until he memorized the environment perfectly. Learning duration of each participant was recorded. For virtual learners, a familiarization phase with the stereoscopic device and the joystick was performed just prior to the learning phase, using a simple L‐shaped virtual building, different from the learning environment and without any landmark.

Experimental conditions (inside the MR scanner)

The day following the learning phase, both groups of subjects performed three scanning sessions, each including a distance comparison task and a baseline condition (described below). Each of the three sessions included four blocks of four distance comparisons each alternated with five blocks of baseline. These blocks were counterbalanced across participants so as to control for presentation order effects. Both conditions were performed with eyes closed in total darkness. Just prior to the beginning of the scanning sessions, subjects participated in a training phase consisting in a short session that was identical to the three sessions performed in the MRI scanner (see details below).

Distance comparison task

Participants heard a speaker name two pairs of landmarks belonging to the learning environment and were asked to determine whether the bird‐flight distance formed by the first pair was shorter than the one formed by the second pair, and then to press a “true” or a “false” key (Fig. 1C). When a key was pressed, additional landmark pairs were delivered. The difference between distances was 29% in average (min 14%, max 45%). The number of true and false responses was counterbalanced within each session. Response accuracy and response times were recorded using SuperLab software (Cedrus, San Pedro). Each task block lasted 43.2 s in average and included four comparison judgments. Note that this task required participants to infer a survey view from the ground level perspective they learned. We chose a relative distance comparison task rather than an absolute distance estimation because it is has been suggested that absolute estimations were less accurate in virtual worlds than in real ones [Ruddle et al., 1997]. Such a difference in task difficulty could have biased our interpretation of potential brain activation differences. Moreover despite the realism of the virtual environment, we were concerned by the fact that mental representation coming from real or virtual environment could be different (e.g. more detailed after learning the real environment). The use of a survey representation derived from the route perspective learning to reduce this potential difference. Finally, the relative distance comparison task provided both response times and response accuracy measurements (unlike the classical mental navigation task between two landmarks, for example) and thus allowed a more valid comparison between behavioral data collected in each group [Denis, 2008].

Baseline condition

In this condition, participants were instructed to passively listen to groups of four words (and to refrain from thinking about the distance comparison task) and to press alternately one of two keys at the end of each group. These words came from a list of 10 words that referred to abstract notions and were thus unlikely to result in visual mental image generation: “criterion,” “problem,” “interest,” “temperament,” “doubt,” “hypothesis,” “mistake,” appearance,” “vocabulary,” and “monopoly”. These abstract words matched landmark names for syllable number and familiarity. Each baseline block lasted 30 s and included four groups of words.

Data acquisition

MRI acquisitions were conducted on a GE Signa 1.5‐Tesla Horizon Echospeed scanner (General Electric, BUC, France). The scan session included two anatomical acquisitions and three functional runs. High‐resolution structural T1‐weighted spoiled gradient echo anatomical images (T1‐MRI) were acquired in 128 axial slices (SPGR‐3D, matrix size = 256 × 256 × 128, sampling = 0.94 × 0.94 × 1.5 mm3). Double echo proton density/T2‐weighted anatomical images (PD‐MRI/T2‐MRI) were acquired in 32 axial slices (matrix size = 256 × 256 × 32, sampling = 0.94 × 0.94 × 3.8 mm3). Functional images were acquired in the same slices using a time series of echo‐planar T2* weighted EPI sequence (BOLD, TR = 6 s, TE = 60 ms, FA = 90°, sampling = 3.75 × 3.75 × 3.8 mm3). To ensure signal stabilization, the first three BOLD volumes were discarded at the beginning of each run.

Data preprocessing

The preprocessing was based on the SPM99b subroutines [Ashburner and Friston, 1999; Friston et al., 1995]. The data of each participant were corrected for motion, normalized to the Montreal Neurological Institute T1‐weighted templates [Collins et al., 1994] and smoothed with a Gaussian filter (FWHM = 8 × 8 × 8 mm3).

Data analyses

The functional data were analyzed and integrated in a statistical model by the semi‐automatic software SPM5 (Wellcome Department of Cognitive Neurology, http://www.fil.ion.ucl.ac.uk/spm/).

For each participant, in a first level of analysis, we computed in each subject, three contrast maps that accounted for the BOLD signal, which covaried with the task compared with the baseline. These contrast maps corresponded to the three distance comparison sessions. A second‐level analysis was performed at the group level including the three BOLD contrast maps for each real learner and the three BOLD contrast maps for each virtual learner. The following contrasts were computed:

  • 1

    Conjunction of mean activation of the three sessions for real learners and for virtual learners under the conjunction null hypothesis [Friston et al., 2005], to determine the activations common to both real and virtual learners (P < 0.05 corrected for multiple comparisons).

  • 2

    Mean activation for virtual learners minus real learners (P < 0.05 corrected for multiple comparisons, masked inclusively by the main effect i.e. mean activation for virtual learners at P < 0.05 uncorrected for multiple comparisons).

  • 3

    Mean activation for real learners minus virtual learners (P < 0.05 corrected for multiple comparisons, masked inclusively by the main effect i.e. mean activation for real learners at P < 0.05 uncorrected for multiple comparisons).

In addition, a functional asymmetries analysis was performed in order to verify a left hemisphere advantage for regions more activated in virtual learners than in real learners. First, voxel‐by‐voxel interhemispheric differences were assessed by subtracting BOLD values of one hemisphere from those of the other hemisphere for each participant and under each condition [Liegeois et al., 2002]. These maps were then entered in a second level analysis and a comparison between virtual and real learners' asymmetry maps was computed (P < 0.05 corrected).

To investigate, the relation between the BOLD signal and the number of correct responses in the hippocampus and parahippocampus we computed a correlation analysis within two anatomical regions of interest encompassing all the parts of these two regions defined on the Montreal Neurological Institute brain template [Tzourio‐Mazoyer et al., 2002].

Post‐test debriefing

A post‐test debriefing was performed outside the scanner. After the fMRI session, participants were asked to describe the strategy they used to solve the task, in particular whether they saw the environment from above, whether they used a mental map of the environment and were asked to score the richness in details of this mental map. We also checked that the distances compared by the participant were actually bird‐flight distances in asking the subjects whether they assessed the distances between landmarks “through the walls” to perform the distance comparison task. Participants were also asked to rate whether mental map they used during the task was detailed or not (from 0: very schematic with no detail, with dots standing for landmarks, to 10: very detailed, landmarks being accurately imaged, and map incorporating elements that were not required to perform the task, including landmarks not concerned by the comparison distance task.). They were then requested to sketch a map of the learned environment, including the position and the name of the landmarks. Similarities and differences between the map drawn by the participants and the actual map of the environment were measured thanks to a distortion index (DI) computed through a bidimentional regression [Friedman and Kohler, 2003].

RESULTS

Post‐Test Debriefing

Five subjects misunderstood the instructions and did not compare bird‐flight distances but the route distances between the landmarks in following the corridor. This way of doing increased dramatically the number of wrong responses, the distances to be compared being not the attended ones. Behavioral data collected during the fMRI session and fMRI data of these five participants were thus removed from the group analyses. The twenty five remaining participants performed correctly the distance comparison task and all formed a mental map of the environment (thus using a survey perspective).

Behavioral Results

Learning phase

The learning duration of the virtual learners (12.9 ± 6.0 min) was not significantly different from that of the real learners (12.2 ± 6.5 min, P = 0.70; Wilcoxon). It was also the case for the number of learnings (Virtual: 1.7 ± 0.6, Real: 1.9 ± 0.5, P = 0.30; Wilcoxon).

Distance comparison task

Response times for the correct responses were not significantly different between virtual (5.1 ± 2.7 s) and real learners (6.8 ± 3.4 s; P = 0.23; Wilcoxon). In addition, percentages of correct responses were not significantly different between virtual (71.1 ± 15.8%) and real learning modalities (71.1 ± 7.4%; P = 0.49; unpaired t test). No correlation was found between MRT and response time or number of correct responses.

Details of the Mental Map

In average, participants reported that the mental map they used to perform the task was poorly detailed (mean 2.8/10; SD: 2.4). Richness in details did not differ between virtual and real learners (P = 0.68, Wilcoxon) and was positively correlated to MRT scores (rho = 0.46, P = 0.01, Spearman).

Map Drawing

All subjects were able to draw map with the landmarks correctly positioned (e.g. the global shape of the environment was respected, no landmarks were omitted and they were placed correctly relatively to each other). However, the bidimensional regression revealed that maps drawn by virtual learners fitted better the actual map than maps drawn by real learners (DI virtual learners: 27.2 SD: 7.9, DI real learners: 34.4 SD 7.6, P = 0.01, unpaired t‐test). Note that this difference between distortion indexes did not result in difference in performances. Indeed, map drawing constituted an indirect behavioral indicator since it required translating an inner representation into a complex motor sequence, outside the scanner. Moreover, the distortion in both groups was slight and did not modify the relative position of landmarks. Accordingly, we found no correlation between the number of correct responses and the distortion indexes (rho = −0.19, P = 0.35, spearman).

fMRI Results

Activation common to both virtual and real learners

Virtual and real learners activated a large common bilateral neural network encompassing frontal, parietal, medial temporal and occipital lobes during the distance comparison task compared with baseline (Fig. 2; Table I). In the frontal lobe, bilateral activations were detected in the precentral sulcus at the intersection with the superior frontal sulcus, extending to most of the premotor cortex, the pre‐SMA, the middle frontal gyrus and the inferior frontal gyrus extending to the insula. In the parietal lobe, we evidenced bilateral activations in the superior and inferior parietal lobules, the intraparietal sulcus and the precuneus. This activation extended to the middle occipital gyrus. Medial temporal activations were found in the parahippocampal gyrus bilaterally at the intersection with the fusiform gyrus. Although the hippocampus was not activated in average, we found, as illustrated by Figure 3, a significant positive correlation between the number of correct responses in the distance comparison task and the activity within the left hippocampus (r = 0.48; P = 0.02) and the right homologue region (r = 0.51; P < 0.009). We also found this correlation within the left Parahippocampus (r = 0.62; P = 0.001) and the right homologue region (r = 0.61; P = 0.001)

Figure 2.

Figure 2

Open in a new tab

Brain areas showing BOLD signal increases in both real and virtual learners during the distance comparison task compared with baseline (conjunction map at P < 0.05 significance level corrected for multiple comparisons), overlaid on the 3D rendering of the MNI template. LH, left hemisphere; RH, right hemisphere.

Table I.

Brain areas commonly activated in both real and virtual learners during the distance comparison task compared with baseline (P 0.05 corrected)

Conjunction of both real and virtual learners
Anatomical localization N voxels X Y Z Z score
Parietal cortex 12,359
 R inferior parietal / intraparietal sulcus 36 −46 44 Inf
 L inferior parietal / intraparietal sulcus −32 −60 52 Inf
 R superior parietal 14 −74 56 Inf
 L superior parietal −14 −70 54 Inf
 R angular gyrus 34 −58 48 Inf
 R precuneus 2 −66 58 Inf
 L precuneus −4 −62 52 Inf
 R supramarginal 42 −38 40 Inf
Occipital cortex
 R middle occipital 42 −76 32 Inf
 L middle occipital −30 −74 34 Inf
 L cuneus −10 −76 38 7.81
Temporal cortex
 R middle temporal 50 −66 18 7.36
Subcortical areas
 R thalamus 1818 8 −14 12 Inf
 L thalamus −8 −12 12 Inf
 R caudate nucleus 20 0 16 Inf
 R pallidum 16 4 0 5,78
 L pallidum −12 6 0 5,46
Cingulate cortex
 Posterior cingulate cortex 18 −54 16 Inf
 Posterior cingulate cortex −16 −58 8 Inf
Frontal cortex
 L inferior frontal pars triangularis/opercularis 5555 −52 14 28 Inf
 R pre‐SMA (R/L) 4 18 50 Inf
 L precentral sulcus / superior frontal −32 0 58 Inf
 R superior medial frontal 4 30 42 Inf
 L middle frontal gyrus −22 8 54 6,78
 R inferior frontal pars orbitaris / insula 625 34 32 −6 Inf
 L inferior frontal pars orbitaris / insula 650 −32 28 −4 Inf
 R middle frontal 1369 28 8 56 Inf
 R superior frontal / precentral 32 2 62 7.46
 R middle frontal 1603 52 40 26 Inf
 R inferior frontal pars opercularis 50 14 28 7,46
 R inferior frontal pars triangularis 54 24 20 5.10
Cerebellar cortex
 L cerebellum 607 −6 −78 −22 Inf
 R cerebellum 10 −78 −22 7.72
 Vermis 2 −54 −24 6.95
Temporal cortex
 R fusiform / parahippocampal 123 28 −38 −14 6.46
 L fusiform / parahippocampal 319 −32 −42 −12 Inf
 L inferior temporal 259 −58 −60 −8 7.22
Open in a new tab

Coordinates of the maximum Z scores are given into the MNI stereotactic space (L: left; R: right). Inf: Z score not computed (P value too close from zero).

Figure 3.

Figure 3

Open in a new tab

Relationship between the number of correct responses and the bold signal variation in left and right hippocampus showing that participants with a great number of correct responses tended to increase their activity in these regions. The absence of significant activation in average is due to the fact that half of the participants had no or negative BOLD signal variation while it was positive in the other half.

Cerebral activity of virtual learners minus real learners

We found that a left‐lateralized fronto‐parietal network was more engaged in virtual learners than in real ones (Fig. 4; Table II). In the frontal cortex, left‐sided activations were observed in the inferior frontal gyrus pars opercularis. In the parietal lobe, we detected leftward activations in the intraparietal sulcus, the superior and inferior parietal lobules, and the precuneus. A left‐sided temporal activation was also noted in the posterior part of the superior temporal sulcus/middle temporal gyrus. The functional asymmetry analysis revealed that leftward asymmetry was significant for all these regions (P < 0.05, corrected). In addition to the results of the present study, Figure 4 reported peak voxel coordinates of previous works dealing with various aspects of tool representation. Our activation in left middle temporal gyrus appeared anterior compared to most studies while we found a good correspondence in inferior frontal and inferior parietal cortex.

Figure 4.

Figure 4

Open in a new tab

Left inferior frontal, inferior parietal, and posterior temporal regions were more activated in virtual learners than in real learners on saggital view of MNI Template. Histograms depict BOLD signal change in virtual (red) and real (green) learners for the three sessions. Colored dots correspond to maximal peak voxel reported by studies on tool perception tool semantic and hand‐tool interaction.

Table II.

Differences between virtual learners and real learners during the distance comparison task compared with baseline

Anatomical localization N voxels X Y Z Z score
Virtual learners minus real learners
Frontal cortex
 L inferior frontal (pars opercularis) 81 −48 12 14 5.62
Parietal cortex
 L intraparietal 61 −26 −68 34 5.46
 L superior parietal 56 −26 −70 56 5.12
 L supramarginal 37 −66 −48 28 5.11
 L precuneus 37 −10 −70 58 4.67
 L inferior parietal 205 −38 −46 40 4.57
Temporal cortex
 L Superior temporal sulcus/middle temporal 29 −50 −42 6 4.36
Real learners minus virtual learners
Frontal cortex
 R caudal LPFC 92 48 16 40 5.11
Parietal cortex
 L occipitoparietal/retrosplenial 43 −16 −76 28 4.85
Open in a new tab

The statistical threshold was set at P corrected < 0.05. and cluster level at 25 voxels. Coordinates of the maximum Z scores are given into the MNI stereotactic space (L: left; R: right).

Cerebral activity of real learners minus virtual learners

The left occipito‐parietal sulcus and the posterior part of the right superior frontal gyrus were more activated in real than in virtual learners (Table II).

DISCUSSION

The main goal of this fMRI study was to investigate whether the real or virtual nature of learning had an impact on the cortical representations of a learned environment. Previous works have suggested that learning an environment by map reading versus actual walking or verbal description led to partially distinct representations in the brain [Mellet et al., 2002]. It is unknown whether it remains true when the two modalities are so similar that they lead to similar performances of the same mental task. In this study, the only difference between the two groups was the nature (real or virtual) of the environment learned the day before the fMRI scanning. In the following discussion, we will further address the similarities and differences at both the behavioral and functional levels between real and virtual learners when they mentally performed an identical distance comparison task.

Behavioral Performances of Virtual and Real Learners

During the learning phase, we found no significant behavioral difference between virtual and real learners in the duration of learning. During scanning, no significant difference was reported in the distance comparison task between learning modalities, neither for the correct response time nor for the percentage of correct responses. After the scanning session, both groups were able to draw accurate maps confirming that they could easily produce a survey representation of the environment. These behavioral similarities are in line with previous works which evidenced broadly similar topographical knowledge between real and virtual environments, at least when simple and immersive virtual environments were used, as is the case in this experiment [Lessels and Ruddle, 2005; Richardson et al., 1999; Ruddle et al., 1997]. In the framework of the present study, this result guaranteed that any brain activation differences between virtual and real learners could not be attributed to differences in performance. A puzzling outcome was that the maps drawn by the subjects after the fMRI session were less distorted in virtual learners than in real learners. It has been shown that when tested in complex real environments (e.g. that included several floors), subjects produced less route knowledge after virtual than after real learning [Witmer et al., 1996]. Note however, that our task required transferring route knowledge to a survey representation. The effect of virtual learning in this context has not yet been studied to our knowledge. Although any interpretation is speculative, this result could reflect a greater flexibility of the topographical representation when no motor representation has been included in the learning. This result has to be taken cautiously and should be reproduced in a larger sample of participants.

Neural Network Shared by Virtual and Real Learners

Both virtual and real learners activated a bilateral parieto‐frontal network, including superior parietal lobules and superior frontal gyri. This network is widely known to be involved in the processing of spatial representations, and constitutes the primary set of regions reflecting the spatial processing in both the virtual and real learning groups [Mellet et al., 2000a]. In addition to this network, occipital activations attested the strong mental imagery component that was part of the task [Mellet et al., 2000b]. Real and virtual learners exhibited bilateral activations in the parahippocampal gyrus. This region has been implicated in the topographical processing of landmarks and spatial scenes, whether the environment explored was real [Epstein and Kanwisher, 1998; Ghaëm et al., 1997] or virtual [Janzen and Van Turennout, 2004]. In this study, the accurate positioning of landmarks is essential in the performance of the mental task, regardless of the way in which the environment was learned. We did not observe any activation in the hippocampus in either type of learning. Inter‐individual variability in performances was likely for the underlying cause of the absence of hippocampal involvement found in the average of the whole group [Hartley et al., 2003; Maguire et al., 1998; Ohnishi et al., 2006]. In this study, this region was activated in half of the subjects (“high performers”) but not activated or even deactivated in the other half (“low performers”).

Overall, when performing the distance comparison task, learners relied on a common network involved in the retrieval and processing of topographical knowledge whatever the environment has been learned through real [Ghaëm et al., 1997; Mellet et al., 2000a] or virtual exploration [Aguirre et al., 1996].

Neural Differences Between Virtual and Real Learners

Although both groups of subjects performed equally well during fMRI scanning, the type of learning affected cerebral activations, in agreement with our hypothesis. Virtual learners exhibited more activity in a left‐lateralized network including the inferior parietal lobule (IPL) and the inferior frontal gyrus (IFG). Critically, this left ventral parieto‐frontal network has also been involved in the execution or the motor imagery of hand movements with a joystick [Stephan et al., 1995], and has been repeatedly implicated in semantic knowledge about actions and tools [Johnson‐Frey, 2004]. More specifically, the left IFG appears to play a key role in conceptual knowledge of tool use [Goldenberg et al., 2007]. This region interacts with the left IPL in hand‐object interactive movement and more generally when visuo‐motor learning is required [Binkofski and Buccino, 2004; Culham and Valyear, 2006]. Both the nature and the left lateralization of these regions are in line with our initial hypothesis: it exists a neural fingerprint of virtual reality related to the use of praxis rather than locomotion during learning. We also reported an activation in MTG, slightly anterior to a region known to be implied in tool processing. This implication encompassed perceptive features such as actual or implicit motion [Beauchamp et al., 2002; Martin et al., 1996], observation of hand movement [Hermsdorfer et al., 2001] and more conceptual aspects such as retrieval of action knowledge [Chao et al., 1999; Kellenbach et al., 2003; Noppeney et al., 2005; Phillips et al., 2002]. It is thus unlikely that this region is involved in a single aspect of tool processing and it might be considered as constituted of several distinct functional areas. In this framework, and considering its anterior shift, further investigations are required to firmly establish that the activation we reported in left MTG is actually part of a “praxis network”.

More generally, this result extends and links together two lines of outcomes. The first line concerns the cognitive operations in which the tools network is activated. To our knowledge, this is the first time that this network was found to be activated in a task that did not involve perception or judgment regarding the manipulability of objects or tools. Rather, in the present study, this set of regions has been embedded in the cortical representation of the environment and manifested itself when this representation was recalled to perform the spatial task. The second line of outcomes concerns the integration of non‐spatial areas in the topographical representation processing network. We previously showed that left temporal and frontal language areas participated in the retrieval and mental scanning of an environment learned verbally [Mellet et al., 2002]. This result is expanded in several ways by this study. First, we observed the activation of non‐spatial areas while the two modalities were very close in term of visual input. This was guaranteed by the use of immersive virtual reality. Second, the behavioral performances of the mental task were similar regardless of the nature of the environment learned, whether real or virtual. Even so, neural differences were detected. Third, the bird‐flight distance comparison task required participants to infer a new topographical representation from the one they initially acquired. Nonetheless, the way in which subjects comprehended the environment has left a neural trace significant enough to be revealed upon activation of the derived knowledge. This suggests that this trace was deeply encoded in the representation and resisted complex transformation. Note that the cortical signature of the virtual nature of the learned environment spared medial temporal regions. This is probably related to the absence of difference in performance between virtual and real learners.

Finally we also reported two regions that were more activated in real than in virtual learners. Although we did not specify any hypothesis regarding the regions more implicated in real learners, it is worth noting that the caudal part of the lateral prefrontal cortex has been sometimes labeled anterior premotor cortex [Badre, 2008]. This term refers to its potential role in abstract motor representation (as compared to motor imagery or to actual motor response) and could be related to the mobilization of spatial knowledge acquired by actual locomotion.

While the activation of retrosplenial cortex (posterior cingulate cortex and parieto‐occipital sulcus) in both groups was expected, its larger activation in real learners was quite unpredicted and difficult to interpret. As a matter of fact it has been suggested that this region played a role in the conversion from egocentric to allocentric perspective [Ino et al., 2002], a process that has no reason to differ between groups. More recently, it has been shown that retrosplenial cortex contribute to the successful formation of survey perspective from a ground level learning [Wolbers and Buchel, 2005]. However, we did not observe any difference in performances between virtual and real learners.

CONCLUSION

While virtual and real learners performed equally well, differences emerged at the neural level. We attributed these differences to the use of praxis (learning through a joystick), for virtual learners, thus reflecting a neural trace of the type of learning. To our knowledge, this is the first study showing that mobilization of topographical knowledge after learning an immersive virtual environment gives rise to a different pattern of activation in the brain than after learning an identical real environment. However, we are aware that although we took great care to match the two environments it is likely that the two types of learning differed in other aspects than the use of a joystick or not. An experiment where participants will learn actively or passively either the real environment or its virtual replication appears necessary and is currently being conducted.

Acknowledgements

The authors are deeply indebted to G. Perchey, G. Pinguet‐Fortin and M.‐R. Turbelin for their invaluable help in volunteer recruitment and data acquisition and to C. Mégard and the team from the Systems and Technology Integration Laboratory (DRT, CEA) for the design of the virtual environment. They also thank A. Shelton for her advice in maps distortion analysis.

REFERENCES

  1. Aguirre GK, Detre JA, Alsop DC, D'Esposito M ( 1996): The parahippocampus subserves topographical learning in man. Cereb Cortex 6: 823–829. [DOI] [PubMed] [Google Scholar]
  2. Ashburner J, Friston KJ ( 1999): Nonlinear spatial normalization using basis functions. Hum Brain Mapp 7: 254–266. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Badre D ( 2008): Cognitive control, hierarchy, and the rostro‐caudal organization of the frontal lobes. Trends Cogn Sci 12: 193–200. [DOI] [PubMed] [Google Scholar]
  4. Beauchamp MS, Lee KE, Haxby JV, Martin A ( 2002): Parallel visual motion processing streams for manipulable objects and human movements. Neuron 34: 149–159. [DOI] [PubMed] [Google Scholar]
  5. Binkofski F, Buccino G ( 2004): Motor functions of the Broca's region. Brain Lang 89: 362–369. [DOI] [PubMed] [Google Scholar]
  6. Chao LL, Haxby JV, Martin A ( 1999): Attribute‐based neural substrates in temporal cortex for perceiving and knowing about objects. Nature Neurosci 2: 913–919. [DOI] [PubMed] [Google Scholar]
  7. Collins DL, Neelin P, Peters TM, Evans AC ( 1994): Automatic 3D intersubject registration of MR volumetric data in standardized Talairach space. J Comput Assist Tomogr 18: 192–205. [PubMed] [Google Scholar]
  8. Culham JC, Valyear KF ( 2006): Human parietal cortex in action. Curr Opin Neurobiol 16: 205–212. [DOI] [PubMed] [Google Scholar]
  9. Denis M ( 2008): Assessing the symbolic distance effect in mental images constructed from verbal descriptions: A study of individual differences in the mental comparison of distances. Acta Psychol (Amst) 127: 197–210. [DOI] [PubMed] [Google Scholar]
  10. Epstein R, Kanwisher N ( 1998): A cortical representation of the local visual environment. Nature 392: 598–601. [DOI] [PubMed] [Google Scholar]
  11. Fogassi L, Luppino G ( 2005): Motor functions of the parietal lobe. Curr Opin Neurobiol 15: 626–631. [DOI] [PubMed] [Google Scholar]
  12. Friedman A, Kohler B ( 2003): Bidimensional regression: Assessing the configural similarity and accuracy of cognitive maps and other two‐dimensional data sets. Psychol Methods 8: 468–491. [DOI] [PubMed] [Google Scholar]
  13. Friston K, Ashburner J, Frith CD, Poline JB, Frackowiak RSJ ( 1995): Spatial registration and normalization of images. Hum Brain Mapp 3: 165–189. [Google Scholar]
  14. Friston KJ, Penny WD, Glaser DE ( 2005): Conjunction revisited. Neuroimage 25: 661–667. [DOI] [PubMed] [Google Scholar]
  15. Ghaëm O, Mellet E, Crivello F, Tzourio N, Mazoyer B, Berthoz A, Denis M ( 1997): Mental navigation along memorized routes activates the hippocampus, precuneus, and insula. Neuroreport 8: 739–744. [DOI] [PubMed] [Google Scholar]
  16. Goldenberg G, Hermsdorfer J, Glindemann R, Rorden C, Karnath HO ( 2007): Pantomime of tool use depends on integrity of left inferior frontal cortex. Cereb Cortex 17: 2769–2776. [DOI] [PubMed] [Google Scholar]
  17. Hartley T, Maguire EA, Spiers HJ, Burgess N ( 2003): The well‐worn route and the path less traveled: Distinct neural bases of route following and wayfinding in humans. Neuron 37: 877–888. [DOI] [PubMed] [Google Scholar]
  18. Hermsdorfer J, Goldenberg G, Wachsmuth C, Conrad B, Ceballos‐Baumann AO, Bartenstein P, Schwaiger M, Boecker H ( 2001): Cortical correlates of gesture processing: Clues to the cerebral mechanisms underlying apraxia during the imitation of meaningless gestures. Neuroimage 14: 149–161. [DOI] [PubMed] [Google Scholar]
  19. Ino T, Inoue Y, Kage M, Hirose S, Kimura T, Fukuyama H ( 2002): Mental navigation in humans is processed in the anterior bank of the parieto‐occipital sulcus. Neurosci Lett 322: 182–186. [DOI] [PubMed] [Google Scholar]
  20. Janzen G, Van Turennout M. ( 2004): Selective neural representation of objects relevant for navigation. Nat Neurosci 7: 673–677. [DOI] [PubMed] [Google Scholar]
  21. Johnson‐Frey SH ( 2004): The neural bases of complex tool use in humans. Trends Cogn Sci 8: 71–78. [DOI] [PubMed] [Google Scholar]
  22. Kellenbach ML, Brett M, Patterson K ( 2003): Actions speak louder than functions: The importance of manipulability and action in tool representation. J Cogn Neurosci 15: 30–46. [DOI] [PubMed] [Google Scholar]
  23. Lessels S, Ruddle RA ( 2005): Movement around real and virtual cluttered environments. Presence 14: 580–596. [Google Scholar]
  24. Liegeois F, Connelly A, Salmond CH, Gadian DG, Vargha‐Khadem F, Baldeweg T ( 2002): A direct test for lateralization of language activation using fMRI: Comparison with invasive assessments in children with epilepsy. NeuroImage 17: 1861–1867. [DOI] [PubMed] [Google Scholar]
  25. Maguire EA, Burgess N, Donnett JG, Frackowiak RS, Frith CD, O'Keefe J ( 1998): Knowing where and getting there: A human navigation network. Science 280: 921–924. [DOI] [PubMed] [Google Scholar]
  26. Martin A, Wiggs CL, Ungerleider LG, Haxby JV ( 1996): Neural correlates of category‐specific knowledge. Nature 379: 649–652. [DOI] [PubMed] [Google Scholar]
  27. Mellet E, Bricogne S, Crivello F, Mazoyer B, Denis M, Tzourio‐Mazoyer N ( 2002): Neural basis of mental scanning of a topographic representation built from a text. Cereb Cortex 12: 1322–1330. [DOI] [PubMed] [Google Scholar]
  28. Mellet E, Bricogne S, Tzourio‐Mazoyer N, Ghaëm O, Petit L, Zago L, Etard O, Berthoz A, Mazoyer B, Denis M ( 2000a): Neural correlates of topographic mental exploration: The impact of route versus survey perspective learning. NeuroImage 12: 588–600. [DOI] [PubMed] [Google Scholar]
  29. Mellet E, Kosslyn SM, Mazoyer N, Bricogne S, Denis M, Mazoyer B ( 2000b): Functional anatomy of high resolution mental imagery. J Cogn Neurosci 12: 98–109. [DOI] [PubMed] [Google Scholar]
  30. Naito E, Ehrsson HH ( 2006): Somatic sensation of hand‐object interactive movement is associated with activity in the left inferior parietal cortex. J Neurosci 26: 3783–3790. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Noppeney U, Josephs O, Kiebel S, Friston KJ, Price CJ ( 2005): Action selectivity in parietal and temporal cortex. Brain Res Cogn Brain Res 25: 641–649. [DOI] [PubMed] [Google Scholar]
  32. Ohnishi T, Matsuda H, Hirakata M, Ugawa Y ( 2006): Navigation ability dependent neural activation in the human brain: An fMRI study. Neurosci Res 55: 361–369. [DOI] [PubMed] [Google Scholar]
  33. Phillips JA, Noppeney U, Humphreys GW, Price CJ ( 2002): Can segregation within the semantic system account for category‐specific deficits? Brain 125: 2067–2080. [DOI] [PubMed] [Google Scholar]
  34. Richardson AE, Montello DR, Hegarty M ( 1999): Spatial knowledge acquisition from maps and from navigation in real and virtual environments. Mem Cognit 27: 741–750. [DOI] [PubMed] [Google Scholar]
  35. Ruddle RA, Payne SJ, Jones DM ( 1997): Navigating buildings on “desktop” virtual environments: Experimental investigations using extended navigation experience. J Exp Psychol Appl 7: 143–159. [Google Scholar]
  36. Stephan KM, Fink GR, Passingham RE, Silbersweig D, Ceballos‐Baumann AO, Frith CD, Frackowiak RS ( 1995): Functional anatomy of the mental representation of upper extremity movements in healthy subjects. J Neurophysiol 73: 373–386. [DOI] [PubMed] [Google Scholar]
  37. Tzourio‐Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, Delcroix N, Mazoyer B, Joliot M ( 2002): Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single‐subject brain. NeuroImage 15: 273–289. [DOI] [PubMed] [Google Scholar]
  38. Vandenberg SG, Kuse AR ( 1978): Mental rotations, a group test of three‐dimensional spatial visualization. Percept Mot Skills 47: 599–604. [DOI] [PubMed] [Google Scholar]
  39. Witmer B, Bailey J, Knerr B, Parsons K ( 1996): Virtual spaces and real world places: Transfer of route knowledge. Int J Hum‐Comput Studies 45: 413–428. [Google Scholar]
  40. Wolbers T, Buchel C ( 2005): Dissociable retrosplenial and hippocampal contributions to successful formation of survey representations. J Neurosci 25: 3333–3340. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Human Brain Mapping are provided here courtesy of Wiley

RESOURCES