Turning Virtual Reality into Reality: A Checklist to Ensure Virtual Reality Studies of Eating Behavior and Physical Activity Parallel the Real World

Abstract

Virtual reality (VR) provides a potentially powerful tool for researchers seeking to investigate eating and physical activity. Some unique conditions are necessary to ensure that the psychological processes that influence real eating behavior also influence behavior in VR environments. Accounting for these conditions is critical if VR-assisted research is to accurately reflect real-world situations. The current work discusses key considerations VR researchers must take into account to ensure similar psychological functioning in virtual and actual reality and does so by focusing on the process of spontaneous mental simulation. Spontaneous mental simulation is prevalent under real-world conditions but may be absent under VR conditions, potentially leading to differences in judgment and behavior between virtual and actual reality. For simulation to occur, the virtual environment must be perceived as being available for action. A useful chart is supplied as a reference to help researchers to investigate eating and physical activity more effectively.

Introduction

Virtual reality (VR) is commonly seen as a convenient way to study a variety of health-related behaviors. Virtual reality allows researchers to construct environments and conditions that would be difficult to realize in a real-world study. This can allow for greater efficiency, safety, and flexibility—all at a lower cost. Accordingly, VR has been used to study various aspects of human behavior and judgment, including obesity and eating disorders,¹ improving eating habits,² health behaviors,^3–5 and shopping behavior.⁶

The reliance on VR to study human behavior and judgment assumes that VR parallels reality to a sufficient extent, such that any findings unveiled in VR conditions would be similar to those that would be found in the real world. However, the ostensible similarity of a VR environment and a real-world environment may obscure meaningful differences in evoked psychological processes, which might, in turn, lead to differences in judgment and behavior, rendering findings obtained with VR as potentially inapplicable to real life.

The factors that are most critical in translating VR simulations into accurate analogues of eating behaviors and physical activity are, at this point, not fully understood. This article addresses some of the factors that would lead VR research to be reflective of real-world outcomes, centering on the process of mental simulation of future actions and experiences.

To parallel the real world, a VR simulation for eating behavior and physical activity must successfully operate through the psychological process of mental simulation in all cases where mental simulation is involved in the parallel real-world setting. Simulation is a crucial component in much of real-world human judgment and behavior.^7,8 Consequently, researchers need to ensure that the conditions support its presence as they would in a real-world environment.

We discuss the importance of simulation in judgment and follow up with (1) a discussion of the reasons why simulation may not be evoked in VR where it would occur in real-life and (2) a checklist of what criteria are most critical in obtaining effective VR simulations of eating behavior and physical activity. We outline conditions under which this absence may lead judgment and behavior in VR to differ from those that would occur in reality and conclude with recommendations for the manner in which researchers may increase the extent to which a VR environment would yield findings that would apply in (nonvirtual) reality.

Mental Simulation in Judgment

Judgment and behavior are often determined by people’s projections of future actions and events.^9,10 The human brain commonly simulates actions and events as part of its routine psychological functioning.⁷ Such simulations can be conscious but are often nonconscious and unintended, automatically triggered by the environment and running to completion without the person’s awareness.⁸ Such simulations, whether conscious or not, may influence people’s perception and judgment of their environment and may, in turn, guide behavior.¹¹

Mental simulations are grounded in a particular setting and incorporate elements from the person’s current situation.⁷ For example, simulation of walking would incorporate the terrain in the person’s environment and the person’s current physical condition (e.g., level of fatigue). This crucial characteristic of simulations can affect the manner in which people perceive and choose to navigate their environment. Environmental elements, as well as the person’s current physical state, are incorporated in simulations. Simulations are consequently altered in a manner similar to how real-world action or experience would be altered by such situational elements. For instance, a difficult terrain and heightened fatigue would cause a simulated walk to feel more effortful and, consequently, longer. In this manner, simulations come to reflect how the current internal or external situation would affect the experience of an object or action. To the extent that current situational elements would alter a person’s experience or perception, they would also alter the course of simulated action (e.g., walking, eating) and, subsequently, the manner in which an element in the environment is perceived, judged, and acted upon. If a person is hungry, for example, simulation of eating will make food appear tastier and potentially lead to greater purchase and consumption.

Mental simulation has been shown to alter judgments of the possibility for action,¹² duration required for action,^13,14 and physical dimensions of the environment.^15,16 For instance, carrying a heavy load makes distances appear longer in peoples’ simulations of crossing a distance and, consequently, leads to distance extension.¹⁵

Research has shown the links between the presence of simulation and judgment of food products, with potential implications for food choices and health (unpublished data). In this research, participants carrying heavy backpacks made judgments of various product attributes relating to perceived product size. Participants carrying heavy backpacks judged products to require less storage space (2.49 versus 3.37 on a nine-point scale): F(1, 12) = 4.7, p = .05. In a replication study, burdened participants also judged products to fit in smaller bowls [average bowl size dropped from 3.2 to 2.98 on a five-point scale; F(1, 26) = 6.81, p = .01] and weigh less [10 oz instead of 11.27 oz per unit; F(1, 26) = 4.74, p = .04]. All these effects consistently indicate that food items were perceived to be smaller and burdened participants felt that they would need a greater quantity of each item. This is because, when carrying a heavy backpack, people require more energy for any action. Because of this, products subjectively supply less energy; in other words, they are subjectively smaller. Such a reduction in experienced energy input would become apparent in a simulation of the experience of eating.

Judgment in conditions where simulation occurs thus leads to distortion where objects are perceived in terms of their current affordances, what they allow the person to do in his current state and situation. In this case, simulation leads food, which is used to provide energy, to be perceived as smaller because it subjectively provides lower energy given the participants’ burdened state. This may, in turn, prompt burdened participants to buy more food.

Related work has demonstrated that such effects do indeed depend on the existence of simulation. Thus distance judgment expanded because of heavy burden only for participants in circumstances that supported simulation.¹⁵ Similar results were obtained by Witt and Proffitt.¹⁶ The research also outlined the conditions required for simulation to occur, as described in the next section.

Consistent with the energy explanation and with the dependence of such effects on simulation, a separate line of studies had participants judge the hours of energy provided by energy drinks.¹⁷ Participants carrying heavy backpacks estimated that drinks would provide less energy. Again, this occurred because, subjectively, each drink would provide less energy because more energy is required for any action when a person is burdened. Importantly, these effects occurred only in circumstances where energy drinks were visible and reachable to participants—both conditions for the occurrence of mental simulation.

Given the effects of simulation, certain conditions are necessary for mental simulation to occur and must be taken into account in the use of VR to study people’s judgement and behavior. Ensuring the presence of such conditions in domains where simulation occurs in real-life settings would ensure the practical applicability of VR research.

Conditions for Mental Simulation in Virtual Reality

The occurrence of mental simulation depends on the presence of several conditions. As discussed earlier, visibility of targets (or vivid mental representations of the items) is one such crucial condition. While this condition is amply fulfilled in VR, other crucial conditions might not be.

Visibility of an object can be subsumed under a broader class of elements required for simulation to occur. Specifically, the environment needs to be perceived as immediately actionable. In other words, for the brain to automatically simulate action, the person needs to experience particular elements of the environment as available for immediate action.^15,17 Either vivid representations or immediate presence of the object might suffice for this to occur.¹⁸

While the virtual environment might provide a good visual approximation of reality, if it is not also perceived as psychologically real, simulation would not be triggered. For simulation to occur, research participants need to perceive that objects in the virtual environment are truly “actionable”—available for immediate action—or, alternately, have sufficient experience with particular stimuli that would supply them with sufficiently vivid action representations. Even in this latter case, immediate presence of an object perceived as actionable might be a better prompt for simulation to occur.

To illustrate this idea, consider the instance of food. In the energy drinks studies cited earlier, participants who could not see the drinks during judgment did not demonstrate the deflated energy judgments shown by those who could see the drinks during judgment.¹⁷ This occurred despite the fact that the drinks were shown to them in a picture before making their judgments. Ostensibly, some representational forms do not suffice to trigger simulation: objects must be perceived as sufficiently real at the time of judgment for the mind to automatically engage in simulation. If no sufficient experience exists to generate sufficiently vivid representa-tions, objects in the environment need to be such that they are perceived as actionable. This might be particularly true in the instance of food, given the elaborate nature of engagement and its multisensory properties.

As noted, sufficiently vivid representations may suffice to trigger simulation, substituting for direct vision.¹⁸ Some objects might be perceived as “real enough” for the brain to engage in simulated interaction in that they trigger preexisting object and action representations. For instance, a virtual image of a rotating cup has been shown to suffice for triggering grasping simulation.¹³ This opens the door for object representations in VR to trigger simulation. However, the precise lines beyond which an object is not perceived as real or sufficiently vivid have not been carefully studied and need to be delineated if researchers are to design virtual environments where the brain is led to perceive objects as real enough to stimulate simulated interaction.

When it comes to food, such “credibility” of objects might involve senses other than sight alone. Research in the domain of addiction has, in fact, revealed that the presence of sensory cues such as olfaction contributes to the credibility of the virtual environment, enhancing people’s sense of “presence” in the virtual environment.^3,19 Inclusion of such elements that enhance an environment’s credibility may help make a virtual environment a better proxy for actual real-world environment, allowing translation of VR findings to real life, for instance, in the treatment of craving and substance abuse.

Creating Psychologically “Credible” Virtual Environments

Virtual reality is generally used to provide a convenient means of studying a vast array of health-related behaviors. In some domains, sufficiently credible environments have been developed so as to allow translation of VR behaviors to the real world. For instance, researchers working with VR simulations of surgical operations have found that training in a virtual environment can lead to improved performance in actual surgery.^20–22

However, as indicated earlier, care needs to be taken to ensure that virtual environments provide a close parallel of real-world environments so that the psychological processes leading to people’s judgment and behavior would occur in the virtual environment as well. These elements may change on a case-by-case basis, depending on the psychological processes specific to each domain. For instance, the importance of cue-reactivity in alcohol consumption may imply that the presence of sensory cues other than vision is important for the studying and VR-supported altering of such behaviors.³

During the workshop discussion on the role of VR in studying health behavior, which served as a basis for several articles featured in this issue of Journalof Diabetes Science and Technology, it was said that “visual presentations could assist subjects in adjusting distorted assessments of portion sizes, correcting unrealistic expectations of the rate of weight loss, managing adverse sensory experiences from behavior change.”²³ However, as argued earlier, such effects and interventions developed to address them might differ between a virtual environment and a real-world environment unless care is taken to ensure that the virtual environment is perceived as sufficiently actionable to trigger simulated action. Researchers need to keep in mind the factors required for the triggering of mental simulation when designing a virtual environment. If objects in the virtual environment are not perceived as being available for immediate action, people may not simulate acting on these objects and would not “experience” the effects of current state on how an object is judged and experienced, causing what occurs in VR conditions to be different from what would happen in a parallel real-life situation.

A useful table that can be helpful in making sure a VR situation closely parallels reality is given here. Some of the checklist items in Table 1 have been empirically supported in VR research.^3,20 Other items, such as those reported in our findings, are just emerging. However, these would all be critical determinants of how successful VR studies would be in mapping onto actual eating, physical activity, and other behaviors.

Table 1.

Checklist of Crucial Factors in Turning Virtual Reality into Reality

The person’s physical state and what the person is wearing and holding and the manner in which they are positioned in the virtual environment should all parallel reality. For instance, holding a VR controller might interfere with simulation of grabbing.

Environments must be designed so that participants feel that all pertinent objects are available for interaction. Objects in a virtual environment must be perceived as available for actual action and manipulation.

All relevant senses need to be involved in a VR environment, not just sight, but smell, sound, and touch (textures).

Care must be taken that irrelevant externalities (laboratory conditions, VR equipment, people in the actual environment) do not interfere with the VR environment.

Attention needs to be isolated to the focal simulation object rather than to other irrelevant factors inside the simulation, such as unusual-looking people or unfamiliar noises, unless such elements would have parallel effects in a real-world environment.

Allocation of attention must be similar to that in a parallel real-world situation, without undue attention being focused on focal study objects.

Care must be taken to ensure cognitive involvement of the participant matches the level of involvement in the actual situation.

Conditions in the environment must parallel those of reality since even peripheral elements might interact with focal elements in simulation to determine outcomes.

When studying habits, all triggering cues upon which habits are dependent must be available since habits are context dependent.

The spatial characteristics of the virtual environment must be taken into account (e.g., when studying spatial distortions, for instance, two-dimensional versus three-dimensional).

Limitations and Future Research

The current research focuses on one main instance involved in the triggering of simulation. Specifically, we focused on differences between conditions of “actionability” perception in a virtual environment and those in an actual environment. Other factors might also lead to either the absence of presence of mental simulation or to the alteration of such simulation in a manner that would affect judgment or action. Additional inconsistencies between a virtual environment and a real-life environment that would affect the course of simulation and consequently research findings could involve, for instance, what one is holding, the ambient temperature, and sound conditions.

One main factor that affects the course of mental simulation is the physical condition of participants. Mental simulation meshes the person’s physical state and the environment and reflects the manner in which the two would interact in the person’s actions and experiences within the environment. Accordingly, one avenue where VR holds great promise in the study of exercise and eating behavior lies in the actual manipulation of a physical state (such as hunger or fatigue) coupled with use of environmental manipulations that could not be easily studied in real life. This would help researchers determine how a person in a particular physical state would respond while being in a difficult environment.

To illustrate this idea, consider how people physically denied food because of a natural disaster or a personal emergency judge food and make food choices. Food deprivation might alter food choices and consumption behavior (Wansink and Tal, paper under review). Such effects of physical states on people’s food judgment and choice clearly involve the intersection of physical state and situation (e.g., what foods are available, how the environment lends itself to obtaining food, what elements in it hamper or assist in obtaining food).

Virtual reality research allows us to safely manipulate physical state while overlaying it into an unsafe situation that cannot be naturalistically studied. For instance, in studying how people’s eating choices change after having gone through a period of deprivation, we can simulate a starvation environment where such physical states might occur.

Conclusion

In this article, we argued that use of VR in research may lead to biased findings regarding judgment and behavior when it does not fulfill the conditions required to trigger the psychological process of mental simulation. Simulation is involved in the determination of many real-world judgments and behaviors. It occurs naturally in real-world situations but may be absent in VR unless care is taken to ensure VR fulfills the necessary conditions for the occurrence of simulation.

Specifically, in order to trigger simulation, objects and environments in VR need to be perceived as real enough to allow action, particularly in situations where no extensive experience exists to allow reliance on preexisting mental representations. For example, food in VR needs to feel sufficiently real or similar to real-world food to make research participants feel that they could eat it. A bar environment may need to feel similar to an actual environment to trigger similar actions.³ Otherwise, simulation may not occur or may occur more weakly or less often than it would in an actual environment, and the findings of VR research about food judgment and choice would not extend to real-world situations, where simulation is commonly involved.

Mental simulation is just one important instance of a psychological process that might differ between VR and real-world situations and so lead to meaningful differences in judgments and behaviors. Other psycho-logical processes underlying judgment might be altered in a virtual environment. In each domain, care should be taken to ensure VR offers all those environmental elements that serve as the basis of relevant processes. Future research should thus be dedicated not only to more precisely studying the boundary conditions for the occurrence of simulation, but also to the study of other psychological processes that might be different in a virtual environment versus a real environment.

Abbreviations

(VR)

virtual reality