Virtual Reality and the Role of the Prefrontal Cortex in Adults and Children

Abstract

In this review, the neural underpinnings of the experience of presence are outlined. Firstly, it is shown that presence is associated with activation of a distributed network, which includes the dorsal and ventral visual stream, the parietal cortex, the premotor cortex, mesial temporal areas, the brainstem and the thalamus. Secondly, the dorsolateral prefrontal cortex (DLPFC) is identified as a key node of the network as it modulates the activity of the network and the associated experience of presence. Thirdly, children lack the strong modulatory influence of the DLPFC on the network due to their unmatured frontal cortex. Fourthly, it is shown that presence-related measures are influenced by manipulating the activation in the DLPFC using transcranial direct current stimulation (tDCS) while participants are exposed to the virtual roller coaster ride. Finally, the findings are discussed in the context of current models explaining the experience of presence, the rubber hand illusion, and out-of-body experiences.

Introduction

Presence is a relatively new concept that, with the advent of computer technology, has acquired considerable importance in the effective design of virtual environments that simulate real world events and settings. Presence is understood to refer to the subjective feeling of being in a virtual environment while being transiently unaware of one's real location and surroundings and of the technology that delivers the stream of virtual input to the senses (Vorderer et al., 2004; Wirth et al., 2007). The notion of “spatial” presence emphasizes on the perception of spatial cues in a mediated environment and the experience of presence such that the person is concurrently “inattentive” to the spatial cues of the real physical surroundings. Television, radio, and even books, like virtual reality (VR) technology, facilitate a sense of spatial presence in different ways (IJsselsteijn et al., 2000).

An alternative view on presence proposed by Sanchez-Vives and Slater (2005) highlights the role of supported actions in the (real or virtual) environment as a constituent feature of the experience of reality. The view is elegantly expressed by Sanchez-Vives and Slater (2005, p. 333) that “the key to the approach is that the sense of ‘being there’ in a virtual environment (VE) is grounded on the ability to ‘do there’”. In our view, this ability does not mean that real actions must be executed in a VE. A mental representation of an action can be automatically triggered by the incoming VE stimuli irrespective of subsequent execution of the action or not. Several brain imaging studies have demonstrated that the motor cortex (primary and secondary) can be activated by various stimuli (including VE stimuli) without resulting in overt actions (Baumgartner et al., 2007; Munzert et al., 2009). Hence, the motor representations of the upcoming movements strongly associated with a particular stimuli are most likely used to shape perception along with the experience of the stimuli and in particular the perception and experience of the VE.

Although the experience of presence in a VE is a common phenomenon, the conceptualization and measurement of the presence are not easy. The assessment of presence is most often approached on the basis of post-test questionnaires and rating scales specifically constructed to measure subjective presence experience (Witmer and Singer, 1998). Physiological measurement of heart rate, skin conductance, respiration rate, and peripheral skin temperature enables continuous assessment of bodily responses during exposure to VR environments, and such indices of subjective responsiveness to virtual stimulation have been especially useful in environments that would elicit arousal in the real world (Baumgartner et al., 2006; Meehan et al., 2005; Wiederhold et al., 2000). Physical activity during exposure to virtual stimulation has also been registered and used as an indicator of presence. For example, changes in posture in response to spatial cues and observed motion in a virtual environment may be taken as a sign of presence (Freeman et al., 2000). An alternative strategy for the measurement of presence is based on the idea of “breaks in presence” (BIP). A BIP can occur at the moment of attentional disengagement from the virtual display and during “return” to reality in response to an external stimulus or an internal stimulus (Slater and Steed, 2000). The change of subjective feeling associated with a BIP can be registered and taken as evidence of presence.

The extensive usage of self-reports of presence reflects the importance of evaluating immediate subjective experience of presence. The reduced reliability of such presence measures (Freeman et al., 1999) reflects, in part, the difficulty in both retrospectively and explicitly characterizing the actual contents of experience and the reliance on the perceptual awareness of presence. One promising avenue for understanding presence uses neuroimaging methods, but only a very few studies have explored the neural underpinnings of presence. How can such studies contribute to the understanding of presence? Firstly, the studies can uncover the neural networks involved in generating and modulating presence. Secondly, the studies can demonstrate inter-individual differences in the neural activation pattern of subjects who report differences in the presence experience. Thirdly, it is also possible to track the changing activation patterns observed during the exposure to VR scenarios, during the course of learning and habituation, or as a consequence of specific prior experience. Finally, once the network involved in presence is delineated, the activation of particular nodes of this network can be selectively modulated (using transcranial magnetic stimulation: TMS or transcranial direct current stimulation: tDCS) to determine a specific role of a particular node in modulating the presence experience. The studies that have until now helped to understand the neural underpinnings of presence are summarized as follows.

Involvement of a Network in the Modulation of Presence

Using a virtual roller coaster scenario, Baumgartner et al. (2008) measured the hemodynamic responses with fMRI in a large sample of adults and children. Baumgartner and colleagues also registered the subjective experience of presence after each roller coaster ride. It is worth mentioning that they used different scenarios to control color perception, stereopsis, and dynamic aspects of visual perception. Consequently, they uncovered a distributed network, which is active during roller coaster scenarios that are correlated with strong presence experience compared with the scenarios that are correlated with weaker presence experience. The network comprises extra-striate areas, the dorsal visual stream, the superior parietal cortex (SPL) and inferior parietal cortex (IPL), parts of the ventral visual stream, the premotor cortex (PMC), and the brain structures located in the basal and mesiotemporal parts of the brain (see also Figure 1 for a summary).

Figure 1.

Demonstration of brain areas that are more strongly activated during the presentation of a roller coaster scenario that evokes high presence versus low presence. (A) Increased hemodynamic responses overlaid on a three-dimensional (3D) rendered brain and two sagittal brain slices. (B) Schematic depiction of the stronger activated brain areas during the high presence condition.

However, the network was differently activated depending on the strength of the presence feeling. It is found that the dorsolateral prefrontal cortex (DLPFC) played a pivotal role in the control of the network and the associated experience of presence. Firstly, the hemodynamic responses in the right-sided and left-sided DLPFC showed a strong negative correlation with the subjective experience of presence during the roller coaster rides (Figure 2). Hence, the stronger the presence experience, the lesser was the hemodynamic response in the DLPFC (left and right). By using effective connectivity analyses the authors identified a specific role of the right-sided DLPFC in the control of the experience of presence and the concomitant hemodynamic responses in the network by using effective connectivity analyses. A “negative connectivity” between right-sided DLPFC activation and brain areas was found in the dorsal visual stream, extra-striate areas, the SPL and the IPL, and in the PMC (Figure 3). Based on this finding, we indicate that the right-sided DLPFC down-regulates the activation in the dorsal visual processing stream. Considering the specific role of the dorsal stream in egocentric processing of the visual environment, it can be proposed that the right DLPFC is recruited as part of a strategy for regulating the experience of presence by constraining the egocentric processing of the roller coaster stimulus display. It can also be proposed that by increasing the activation in the dorsal visual stream during strong presence experience (with diminished activation in the right-sided DLPFC), the brain attentively prepares actions in the virtual environment as if the brain actually responds to real-life situations. It is known that the dorsal visual stream and the connected parieto-frontal areas are strongly involved in action and movement control. Hence, the stronger the participants are involved in the virtual scene, the stronger they plan to act attentively in the virtual environment.

Figure 2.

Correlations between subjective presence experience and the hemodynamic responses in the right-sided (A) and left-sided (B) DLPFC. The correlations were calculated between the mean hemodynamic response in the DLPFC and the subjective presence measures.

Figure 3.

(A) Network, which is down-regulated (in blue) or up-regulated (in red) by the right-sided or left-sided DLPFC in adults. The down-regulated brain areas show “negative connectivity” with the right-sided DLPFC implying that strong activation of the right-sided DLPFC is associated with reduced activation in the network (and vice versa). The up-regulated brain areas show “positive connectivity” with the left-sided DLPFC. (B) The network shows “positive connectivity” with the right-sided DLPFC and “negative connectivity” with the left-sided DLPFC in children. Actually, there is no down-regulation but rather an up-regulation by the right-sided DLPFC in children.

In addition to the right-sided DLPFC, there was some influence from the left-sided DLPFC in the form of positive connectivity with the medial PFC (including the ACC), the thalamus and brainstem areas, the ncl. caudatus, and the parahippocampal area. Considering the negative correlation between the presence experience and the activation in the left-sided DLPFC, strong activation of the areas is found in situations in which participants experience less presence, which means that less presence experience is associated with increased activation in the associated brain areas. The medial PFC is known to be functionally involved in attention modulation, conflict monitoring, cognitive control (Botvinick et al., 1999; Carter et al., 1999; Ridderinkhof et al., 2004), and self-referential reflective activity (Amodio and Frith, 2006; Esslen et al., 2008; Gusnard et al., 2001). There have been discussions to ascertain whether the brain areas (especially the medial PFC) are a part of a network involved in the control of the “default mode of brain function” whose activity continues during rest and is suspended during the performance of externally cued tasks (Amodio and Frith, 2006; Greicius et al., 2008; Gusnard et al., 2001; Rilling et al., 2007). It is possible that the recruitment of medial prefrontal regions indicates strong usage by adults of internal self-reflective control processes while being exposed to VE stimuli especially when the adults experience less presence. The subjects appear to control and regulate the presence experience by critically evaluating and monitoring the presented VE stimuli, or by directing their attention away from the external VR to internal self-reflective mental processes.

Interestingly, the above reported pattern of effective connectivity was only found in adults. In stark contrast, children demonstrated a completely different connectivity pattern. In children, the right DLPFC is involved in upregulation of the activation in subcortical and mesiotemporal brain regions (including hippocampus, amygdala, and insula), multi-sensory integration areas (temporo-parietal junction), and areas of the ventral visual processing stream (Figure 3). The completely different activation pattern is understood to indicate a distinctly different “neural” strategy in children for processing the virtual roller coaster scenario. Considering the fact that most of the modulated brain regions in children, particularly the bilateral hippocampus and the areas of the ventral visual stream, are part of a network involved in allocentric, object-based spatial processing, which is independent of the observer's spatial location. Hence, it is suggested that children automatically use the strategy to cope up with a highly immersive virtual roller coaster scenario (Jordan et al., 2004; Nadel and Hardt, 2004). It is known that the allocentric strategy for navigation is typically used in children and adults who are not very well skilled in spatial navigation, due to which the activation pattern might reflect lower proficiency in spatial navigation of children (Mast and Jäncke, 2007). Interestingly, we found an upregulation of activation in the children's brain areas, which are involved in emotional processing (amygdala, hippocampus, and insula). The activation of the areas during arousing VE experience strongly suggests that the children are more susceptible to the arousing impact of the visual and auditory spatial stimuli and are hence less able to regulate and control the experience of presence during confrontation with arousing VE. The differences are most likely related to the prefrontal cortex that is not fully matured (Giedd et al., 1999; Gogtay et al., 2004), which is also associated with delayed maturation of most executive functions in children (Davies, et al., 2004; Segalowitz and Davies, 2004).

Altogether, the presence experience is correlated with a distinct activation pattern in a distributed network. The pivotal node in the network is the DLPFC, with the right-sided part obviously down-regulating the activation in the dorsal visual stream, thus regulating and modulating the presence experience. The left-sided DLPFC is involved in upregulation of the activation of brain areas known to be involved in self-referential action monitoring. In children, there is no evidence of any strong modulatory influence of the DLPFC on the neural network involved in controlling presence. However, more activation was found in mesio-temporal areas including regions known to be involved in emotional processing. The conspicuous absence of any DLPFC influence on the connected brain areas in children most probably depends on the frontal cortex that is not matured, thus undermining the ability of the children to efficiently cope up with the roller coaster scenario, which leads to stronger emotional reactions and a more allocentric orientation in the VR environment.

Influencing Presence by Modulating the Activation of the DLPFC

Earlier studies rely on the measurement and analysis of the BOLD-signal, which is however slow and only correlates indirectly with the underlying electrical neural activation. No direct evidence is found that the activation pattern observed is responsible for true functional sensing of the experience of presence. It is quite possible that the measured activations mostly represent some kind of correlated activations and not necessarily causal activations in the context of presence experience. Selective inhibition or excitation of the area in the context of continuous presence experience would help in determination of the causal influence of a given brain area on the experience of presence. The scientific community is now in a good position to make use of transcranial magnetic stimulation (TMS) or transcranial direct current stimulation (tDCS). This has been done in an experiment in which tDCS was applied to the right DLPFC to modulate the experience of presence while the participants watched a virtual roller coaster ride (Beeli et al., 2008a). During the ride electrodermal activity, impulsiveness and subjective presence experience were measured during exposure to the virtual roller coaster scenario. The application of cathodal tDCS to the right DLPFC (which inhibits the right-sided DLPFC) increased the electrodermal response to the VR stimulus, thus being indicative of increased vegetative arousal. The finding is important as it demonstrates that the emotional responsiveness changed as a function of the right-sided DLPFC activity. With decreased activation of the right-sided DLPFC, the emotional responsiveness increased during the virtual roller coaster ride. However, there was no concomitant increase in the presence experience with the increase in emotional responsiveness. The reason for this unexpected finding might be that the presence questionnaire was filled out after the roller coaster ride, while the electrodermal responses were measured during the ride, and that the retrospective characterization of actual presence experience is hampered by the ongoing physiological effects of DLPFC inhibition. A further possibility is that, in this study, only the right-sided DLPFC was inhibited, although the left-sided DLPFC is also involved in controlling the presence experience. Hence, the inhibition of both the right-sided and left-sided DLPFC may have also modulated the subjective experience of presence. However, the speculation has to be examined in future studies.

Together, the findings indicate the level of significance of the right-sided DLPFC in controlling the actions and the pivotal role of the right-sided DLPFC in the control of the activation of a network that generates or modulates the presence experience. The right-sided DLPFC has been shown to be involved in the control of various behaviors, typically involved in the selection of relevant responses and the suppression of irrelevant responses (Knoch, 2007). For example, it has been shown that driving in a driving simulator is strongly influenced by the activation of the DLPFC, with the DLPFC inhibition resulting in speeded and risky driving, while excitation of the DLPFC (by applying anodal tDCS) leads to a more careful driving style in virtual scenarios without the participants noticing the change in their driving behavior (Beeli et al., 2008b). The finding corresponds with a further study in which driving behavior in the driving simulator was assessed during the measurement of concomitant brain activation. The participants who speeded most and exhibited greater risk-taking behavior during simulated driving demonstrated less neuronal activation in the anterior DLPFC (Jäncke et al., 2008).

A Presence Network?

Did we uncover a “presence network” that is exclusively involved in generating and controlling the presence experience? No, the network mentioned above is not exclusively associated with the modulation of presence experience. It is a network involved in the control of many other psychological functions, including top-down and bottom-up control of attention, spatial orientation, control of egocentric orientation, and the control of motor behavior. The studies mentioned above demonstrate that a particular network is involved in many functions, and the psychological specificity cannot be inferred simply by identifying the activated brain structures. However, this study emphasizes the key role of DLPFC in controlling several behavioral aspects. The DLPFC acts as a modulator of the network and also as a modulator of the concomitant psychological experiences.

Relation of the “Rubber Hand Illusion” with Out-of-Body Experience

The “negative connectivity” in the roller coaster study revealed the influence of the right DPLFC on a frontoparietal attentional network along with the visual cortex is known to show enhanced activity during visual-spatial attention. Of particular interest is the parietal-premotor cortical network because of the role of the network in sensory-motor integration. The posterior parietal cortex (PPC) integrates information from different sensory modalities to form a coherent multimodal representation of space coded in a body-centered reference frame. A feature of the coherent multimodal representation is the representation of peripersonal visual space around the body as found in monkeys (Rizzolatti et al., 1997a) and humans (Halligan and Marshall, 1991; Makin et al., 2007). The integration of multisensory cues around the body in the peripersonal space serves to map the position of objects in the surrounding environment in terms of the own body. The PMC responds only to the movement of objects toward the body parts in the space (Graziano et al., 2002; Rizzolatti et al., 1997a,b), and activity in the ventral PMC of the macaque monkey appears to reflect potential motor actions to targets within the space. Gallese (2005) suggested that such a simulated action enables the definition of motor space represented in terms of potential motor actions to spatial locations.

Jeannerod et al. (1995) suggested that a visual target elicits a motor schema for potential action that maps the position of objects in the surrounding environment irrespective of whether the corresponding action is actually executed (see also Baumgartner et al., 2007). It is proposed that the perception of spatial and motion cues (the human perceptual system is highly sensitive to motion cues) in the mediated environment of the roller coaster elicited, as part of a preparatory motor response, motor schemas that map the fast “approaching” scene in the virtual scenario in terms of real motor space and a corresponding plan for potential action. Such a plan is in effect a simulation of real actions, and as such the “goodness” of the virtual environment may reflect the degree to which the virtual space or a part of the space is represented during presence as a potential and also real motor space, which would effectively put the person “in touch” with the environment.

There are similarities between the experience of presence and the rubber hand illusion (RHI). In the RHI, the subject senses that the rubber hand and not the unseen real hand is the felt location of synchronous tactile stimulation of the hands (Botvinick and Cohen, 1998; Ehrsson et al., 2005). Makin et al. (2008) argued that mechanisms that underlie peripersonal space representation are pivotal for the generation of the RHI. The authors suggested that, if the peripersonal space around the dummy hand is represented as a peripersonal hand space, the visual perception of the stroke of the brush on the dummy hand can be represented in reference frames that are centered on and associated with the dummy hand. It was also suggested that the detection of the correlation between the visual stimulation and the tactile stimulation generates the sense of a single perceptual event. In addition to the proposal, the authors speculate that the “goodness” of the “virtual” (rubber) hand as an illusion may in part reflect the degree to which the “virtual” peripersonal space around the “virtual” rubber hand is represented as a potential and also a real motor space. Attribution of ownership of the rubber hand suggests that there may be an associated representation of the motor space centered on the rubber hand, which would equally apply even when the RHI occurs in the absence of visual stimulation (Ehrsson et al., 2005). A representation of the potential motor space may have the effect of putting the person “in touch” with the space around the virtual hand and of consolidating the sense of ownership.

Recently, several studies have discussed a further phenomenon that is important in the context of the experience of presence, the so-called out-of-body experience (OBE). Several variants of the phenomenon have been described earlier. For example, a completely virtual object (e.g., a hand or an arm) can be experienced as a part of one's self under specific conditions in the virtual environment (Slater et al., 2008). Researchers have shown that the experience of the conscious self can be redirected from the position at which it is normally located (within the body borders) to some other positions in space using specific VR stimuli (Blanke and Metzinger, 2009; Lenggenhager et al., 2007).

OBE has been brought into connection with specific activation in the TPJ (e.g., Blanke and Arzy, 2005; Blanke et al., 2002, 2004; Bünning and Blanke, 2005), a brain region that shows some overlap with the network to be involved in modulating the experience of presence (see also Figure 1). The TPJ (temporoparietal junction) processes multisensory body-related information: involvement of the TPJ and the regions along the intraparietal sulcus in multisensory integration of visual with tactile and proprioceptive information within a common spatial frame of reference (Bremmer et al., 2001; Calvert et al., 2000; Jordan et al., 2001) also in the perception of the body and body parts (Blanke and Arzy, 2005). Together with the posterior insula, the TPJ also forms the central region of the vestibular cortex (Brandt and Dieterich, 1999; Fasold et al., 2002). Electrical stimulation of the posterior insula and retroinsula results in changes in body tilt and the sensation of changed gravitational force (Blanke et al., 2004), and lesions of the areas alter the perception of the visual vertical and rotational vertigo (Brandt et al., 1994). Disturbance of multisensory integration in many VE situations, such as when the graviceptive vestibular sensation of the stationary body is in conflict with the visual impression of changing gravitational forces in the virtual roller coaster, may be coded and re-mapped by the TPJ so as to reconcile the sensory conflict according to the more dominant visual stream of information. If the association is established, the experience of presence is enhanced while a lack of efficient re-mapping might result in “BIP”.

Conclusion

Presence experience evoked by a virtual roller coaster scenario is associated with an increase in activation in a distributed network, which includes the dorsal visual stream, parietal areas, and the PMC. The network is modulated by the DLPFC. The DLPFC activation strongly correlates with the subjective presence experience (the stronger the DLPFC is activated, and the lesser is the presence experience). However, in children, the DLPFC exerts a different modulatory impact on the network. This difference is most likely attributable to the prefrontal cortex that is not fully matured in children and adults.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Key Concept

Presence

Presence is understood as referring to the subjective feeling of being in a virtual environment while transiently unaware of one's real location and surroundings and of the technology that delivers the stream of virtual input to the senses

Biography

Lutz Jäncke is Professor of Neuropsychology at the University Zurich. During the past 25 years he has conducted research on many different topics in cognitive neuroscience, ranging from cognitive neuroanatomy, brain plasticity, sensorimotor control and the neuroscientific basis of music perception and production. Beside these main research branches he has also developed a strong interest in studying the neural basis of presence experience. In addition to his research, he is an active clinical neuropsychologist.