Presence Relates to Distinct Outcomes in Two Virtual Environments Employing Different Learning Modalities

Abstract

Presence in virtual learning environments (VLEs) has been associated with a number of outcome factors related to a user’s ability and motivation to learn. The extant but relatively small body of research suggests that a high level of presence is related to better performance on learning outcomes in VLEs. Different configurations of form and content variables such as those associated with active (self-driven, interactive activities) versus didactic (reading or lecture) learning may, however, influence how presence operates and on what content it operates. We compared the influence of presence between two types of immersive VLEs (i.e., active versus didactic techniques) on comprehension and engagement-related outcomes. The findings revealed that the active VLE promoted greater presence. Although we found no relationship between presence and learning comprehension outcomes for either virtual environment, presence was related to information engagement variables in the didactic immersive VLE but not the active environment. Results demonstrate that presence is not uniformly elicited or effective across immersive VLEs. Educational delivery mode and environment complexity may influence the impact of presence on engagement.

Introduction

For many years, researchers and educators have extolled the promise of virtual environments (VEs), especially immersive ones, as a tool for science education.^1–4 Digital VEs have several purported advantages for the development of teaching tools and platforms for educational research. A key aspect of digital VEs is the elicitation of presence or the psychological immersion of users in virtual learning environments (VLEs), allowing for focused and naturalistic interaction with educational materials and activities.

Presence is often characterized as the central construct defining user experiences in VLEs and in most VEs in general. Presence definitions have proliferated, encompassing multiple psychological and perceptual constructs. Although consensus is lacking in the research community,^5,6 for this work, presence is understood as perceiving as reality the VE as opposed to the physical environment encompassing that VE. Various dimensions of presence can be experienced in learning environments (e.g., physical, social, and self-presence).⁶ Self-presence, “a psychological state in which virtual selves are experienced as the actual self”^6,p46 is the dimension typically discussed in single-user VLEs. The ability of VEs to engender self-presence (hereafter called “presence”) has often been discussed as one of the potential benefits VLEs hold for education.^7–9 Researchers, however, rarely empirically investigate the potential influence of presence on learning and related outcomes in VLEs. Thus, some have called for additional research to examine this relationship.^10,11

A systematic review of the existing literature found that high levels of presence in VLEs among learners were related to better learning outcomes.¹⁰ For example, Winn et al.¹² found that self-reported levels of presence predicted learning in an oceanography simulation. Similarly, examination of a historical simulation VLE revealed that a high sense of presence in the environment helped learners perform assigned educational tasks.¹³

Levels of presence experienced in a VE can vary widely and are related to the form in which information is presented, content of the information, and characteristics unique to the user.¹⁴ Clearly, VLEs vary widely in form (e.g., learning modality) and content (e.g., subject matter) characteristics. In addition to influencing presence levels, configurations of form and content variables in VLEs may also influence how presence operates. These variations in presence should influence downstream learning outcomes. For example, in VLEs with multiple components, users may experience greater presence with respect to certain aspects of a VLE than with other aspects. Consequently, users are more likely to attend to those components than to others. Hence, presence experiences in a VLE may not necessarily lead to increased attention to all environment components, which could differently affect learning of educational content.

Typically, when VLEs are discussed, the focus falls on the technology as a tool for experiential or active learning modalities (i.e., learning via self-driven, interactive activities). Indeed, VE technology’s strong suits are an obvious match for active learning activities. For example, researchers and educators have created and assessed VLEs that immerse learners in an oceanography simulation,¹² interactive worlds with tangible representations of physics concepts,³ botany scenarios in an alien world,⁹ and a historical environment depicting an ancient Greek city.¹⁵

The current state of electronic and online learning, however, still relies heavily on the more traditional didactic modality (i.e., learning though reading or lectures) rather than an active modality.¹⁶ A small number of studies have examined didactic methods using VLEs and immersive VLEs. Bouras and Tsiatsos,¹⁷ for example, focused on a collaborative VLE to immerse learners in a virtual classroom where they could participate in activities including lectures. Bailenson et al.^18,19 systematically studied elements of didactic collaborative VLEs that affect teacher–learner or tutor–learner interaction and learning outcomes. Although didactic methods in VLEs have been infrequently examined in the past, they are being attended to by more researchers. Given the increasing popularity of electronic and online learning, the growing development of VLEs, and the ubiquity of didactic approaches, it appears that instruction represents a likely use of VE and immersive VE technology in the future.

Active versus didactic VLEs exemplify environments in which form and content features are likely to vary in ways that influence presence. Factors related to presence (e.g., interactivity) will often differ in their level of inclusion in active compared to didactic VLEs. Likewise, presence experiences may differ between active and didactic VLEs and consequently affect learning and related variables differently. To assess this possibility, we investigated whether presence operates in different ways in active-based versus didactic-based immersive VLEs to influence engagement-related and learning outcomes as part of a larger study on teaching genetic concepts.²⁰ We compared two VLEs, one based on active and the other on didactic learning, both with the same educational goal: to teach individuals about a genetic concept. We assessed two learning outcomes, six concepts related to engagement with the learning content, presence, and enjoyment of the VLE. The six engagement-related concepts (motivation, interest, attention, elaboration, involvement, and believability) stem from psychological theory²¹ as factors that can influence learning.

Method

Participants

Potential participants were screened to exclude those with epilepsy, vestibular disorders, uncorrected vision or hearing problems, high propensity for motion sickness, and recent employment or training in the field of human genetics. Participants included 156 individuals aged 18 to 40 (M =27.56, SD =6.00) who responded to recruitment advertisements. Participants received $40 for their participation.

Virtual learning environments

Participants were immersed in the VLE via a head-mounted display (HMD)-based immersive VE system wherein their movements in the VLEs were tracked via head position and head orientation and rendered. A handheld wand with position tracking was used to interact with elements of the VLEs. Participants in the active VLE stood and were able to walk around in the VE, whereas participants in the didactic VLE were seated. Equipment included an nVisor SX HMD and a WorldViz PPT X4 tracking system. The VLE software was programmed using the Vizard Virtual Reality Toolkit.

The two VLEs had a common educational objective: to convey that an individual’s genetic makeup interacts with behavioral factors to determine disease risk. Both VLEs were structured around a set of questions intended to prompt learning of the desired concepts.

Both VLEs were based on the metaphor of a “risk elevator,” which moved up or down to represent higher and lower levels of disease risk. This metaphor was determined to be a superior model for representing risk levels during an earlier study.²² A control panel situated within the elevator contained rows of buttons that represented possible levels of three genetic or behavioral risk factors (genetic risk, eating behavior, and exercise behavior). Participants selected combinations of buttons, and the elevator stopped on the floor of a 10-storey building that corresponded to the amount of risk associated with the selected factors. When the elevator stopped on a floor, the doors opened and participants could see 10 digital human representations (i.e., agents) standing in a lobby. A proportionate number of the 10 agents entered a clinic door, providing additional probability-based information analogous to the current level of disease risk. So, for example, if the combination of risk factors chosen via the buttons resulted in a 20% risk for disease, the virtual elevator went to the second floor, and 2 of 10 agents in the lobby entered the clinic.

In the active VLE, participants were able to control the risk elevator directly by using the handheld wand to select buttons and explore different combinations of genetic and behavioral risk factors. These participants were charged with answering a series of learning questions. In the didactic VLE, participants listened to a lecture given by a virtual female health educator in a virtual consultation room. This lecture posed the same questions as were contained in the active VLE, but the questions were answered by the educator using screenshots from the active VLE to illustrate the elevator metaphor. In the didactic VLE, participants were able to raise their hand between segments if they wanted the virtual educator to repeat a segment (Figures 1, 2, and3).

FIG. 1.

View of the active learning virtual environment. Inside the elevator was the control panel. Participants made choices about risk factors using the arrow-shaped selection tool (controlled by the handheld wand).

FIG. 2.

View of the active learning virtual environment. The elevator arrived at the appropriate floor given the levels of risk factors selected, and the elevator doors opened to reveal a waiting room. An appropriate number of agents entered the “clinic” to convey probability information about disease risk.

FIG. 3.

View of the didactic learning virtual environment. The virtual health educator gave participants information about risk levels using the elevator metaphor. She spoke and gestured to a slide show containing scenes depicting the active virtual learning environment.

Procedure

The experimental session lasted approximately 1 hour, of which participants spent about 15 minutes in a VLE. Participants were randomly assigned to complete either the active or didactic VLE. Following the consent process and introduction to the study, participants completed a computer-administered pretest questionnaire. They then listened to an audio file providing instructions for how to operate in their assigned VLE. Once immersed in the VLE, participants had a short warm-up period to adapt to the equipment and subsequently completed the virtual learning task as described previously. Following completion of the task, participants completed a posttest computer-administered questionnaire and were debriefed and dismissed.

Measures

Presence

Users’ experiences of presence in the VLEs were measured by a scale adapted from Swinth and Blascovich.²³ This scale includes eight items averaged into a single measure (full scale available in a previous publication²⁴).

Learning

Learning comprehension was assessed using two distinct measures: information recall and transfer. The recall measure assessed how much information participants remembered (e.g., exercising will lower someone’s chance of getting the disease) using 14 agree/disagree items. Change in recall between pretest and posttest was calculated (to adjust for the effect of guessing) and was analyzed as a change score.

The transfer measure assessed whether participants could apply learned information to new disease contexts. This measure was assessed only at posttest. Transfer was analyzed as the number of items answered correctly out of six.

Learning engagement–related variables

We assessed a number of variables at posttest that are related to learner engagement with educational content. Elaboration of information (i.e., thinking deeply about the information) was based on a 3-item scale adapted from Evland et al.²⁵ Attention to the information and motivation to engage with learning content were each measured by a single item based on Moreno and Mayer.²⁶ Interest (i.e., how interesting the information was) was assessed by a 2-item scale also based on Moreno and Mayer.²⁶ Involvement, or how important participants considered the information to be, was assessed by three items.^27,28 Finally, we assessed believability of the information using a single item.

Enjoyment

Enjoyment of the VLE experience was measured by a 3-item scale adapted from Swinth and Blascovich.²³ Items were averaged into a single measure.

Results

Significance level was assessed as p <0.05 for all analyses. We assessed relationships either using one-way ANOVA or linear regression unless otherwise noted.

Factor analysis

We performed a maximum likelihood factor analysis including all items from the presence and enjoyment scales; the elaboration, involvement, and interest scales; and the three items assessing motivation, believability, and attention. The scree plot indicated a four-factor solution. These factors were rotated using a varimax rotation procedure. The rotated solution (Table 1) yielded the following factors: (1) presence, containing all eight presence scale items; (2) approval of the VLE, containing all three enjoyment items, the motivation item, and the two interest items; (3) information consideration, containing all elaboration items and the attention item; and (4) involvement, containing all three items from the involvement scale. One of the interest items loaded on two factors but was retained in the factor where it had a higher loading. Believability did not load on any of the four factors but was included in further analyses because the purpose of the factor analysis was primarily to evaluate overlap and commonalities between our measures.

Table 1.

Factor Structure of Presence and Learning Engagement Variables

	Factors
Items	Presence	Approval of VLE	Information consideration	Involvement
Forgot I was participating in a study	0.560
Forgot about physical surroundings	0.584
Had to adjust back to physical environment	0.536
Felt like I was in the virtual world	0.714
Wanted to touch things in the environment	0.514
Lost track of time while in virtual environment	0.634
Had sense I returned from a journey when finished	0.793
Virtual environment became reality for me	0.737
Would have liked the experience to continue		0.676
This experience was fun		0.722
Experience in the virtual environment was satisfying		0.830
Would want to use a program like this again		0.643
How interesting was the information?		0.493	0.480
How entertaining was the information?		0.549
How much attention did you pay to the information?			0.632
Found myself focusing on the information			0.907
Tried to relate the information to my own life			0.409
Thought about how the information related to things I know			0.507
How important is effect of genes on health to you?				0.719
How important is information about how genes affect disease risk to you?				0.924
How important is information about how behavior affects disease risk to you?				0.437

Presence

Analysis of the presence scale (α =0.85) revealed a significant difference in presence between the active VLE (M =4.64, SD =1.19) and the didactic VLE (M =3.71, SD =1.25), such that participants reported higher presence in the active VLE, F(1, 154) =16.46, p <0.001.

Presence and learning

We assessed the relationship between presence and learning separately for the active VLE and the didactic VLE (Table 2).

Table 2.

Relationship between Presence and Learning, Learning-Related Variables

	Active VLE (β)	p	Didactic VLE (β)	p
Learning: recall	−0.08	ns	0.02	ns
Learning: transfer	−0.08	ns	0.05	ns
Approval of the environment	0.28	<0.01	0.57	<0.0001
Information consideration	0.08	ns	0.40	<0.01
Involvement	0.05	ns	0.03	ns
Believability	−0.02	ns	0.44	<0.0001

Active VLE

Examining only participants who completed the active VLE, we found no relationship between presence and recall of learning objectives, β =−0.079, p >0.05. We also found no relationship between presence and transfer of learned information to a new context, β =−0.081, p >0.05.

Didactic VLE

Examining only participants who completed the didactic VLE, we found no relationship between presence and recall of learning objectives (β =0.021, p > 0.05). We also found no relationship between presence and transfer of information (β =0.049, p >0.05).

Presence and learning engagement variables

We also assessed the relationship between presence and learning engagement–related variables separately for the active and didactic VLEs using the factors derived from the factor analysis. These findings are displayed in Table 2.

Active VLE

Examining only participants who completed the active VLE, we found no relationship between presence and consideration of the information (β =0.076, p > 0.05), involvement (β =0.047, p > 0.05), or believability β =−0.019, p >0.05). We did, however, find a significant relationship between presence and approval of the VLE (β =.029, p < 0.05).

Didactic VLE

Examining only participants who completed the didactic VLE, we found a significant relationship between presence and consideration of the information (β =0.40, p <0.001). The relationship between presence and approval of the VLE (β =0.57, p <0.001) was also significant, as was the relationship between presence and believability (β =0.44, p <0.0001). The only learning engagement–related variable that was not significantly related to presence in the didactic VLE was involvement (β =0.031, p >0.05).

Discussion

We assessed the role of presence in two VLEs differing in learning modalities but containing the same educational content. One VLE employed active educational methods, and the other used didactic ones. Our findings demonstrated that users of the active VLE experienced greater presence than did users of the didactic VLE. Although presence is suggested to be a major mechanism by which active learning environments improve comprehension, we found no relationship between presence and learning outcomes (recall and transfer) for either type of VLE. Although the active VLE heightened user experiences of presence, it did not produce the anticipated increase in learning. We found no relationship between presence and learning in either type of VLE, but we found that presence was more frequently related to learning engagement variables in the didactic VLE than in the active VLE.

The finding that presence is unrelated to learning outcomes is at odds with the small existing literature.^12,13 However, similar exceptions have been found in the past. For example, Moreno and Mayer⁹ found that use of immersive VLE systems led to increased presence but no gain in retention or transfer. In the current study, the absence of a relationship between presence and learning is relatively robust across two different types of VLEs and across two separate learning outcomes. Also of note is that our sample size was quite large, increasing our statistical power for finding a relationship if one exists.

We believe it is likely that features of our particular VLEs or their content disrupted the anticipated relationship between presence and learning outcomes. This conclusion is consistent with the long-held notion that presence does not operate consistently among VEs but rather varies according to environment features. We therefore suggest that increases in presence would not necessarily increase learning in every VLE.

We did find different patterns in the relationship between presence and variables indicating engagement with learning content. Although presence was related to approval of the VLE in the active VLE, it was completely unrelated to all variables indicating engagement with the learning content (consideration of the information, involvement, and believability). In the didactic VLE, however, we found the anticipated relationship between presence and engagement variables for every construct except involvement.

There is little in the literature that speaks readily to possible reasons for the substantial difference we found in the operation of presence between our VLEs. We suspect that in the simpler didactic VLE, the relationship between presence and information engagement–related variables was clear because being present in the virtual world meant being engaged with the learning material, since there was little else to engage with. Within the active VLE, however, the more complex mode of interaction with learning content may have distorted these relationships. Being present in this virtual world could mean, for example, that participants were engaged with the operation of the virtual elevator and pushing buttons on the control panel to make it go but were not necessarily engaged with the learning content embedded in elevator operation. This interpretation is supported by the finding that presence in the active VLE was highly related to approval, the one factor that applies more generally to the VLE than to the learning content.

Lim et al.²⁹ similarly observed that engagement in an inquiry-based science VLE did not necessarily lead to engagement with learning tasks. For example, they report that on some occasions students were distracted by elements of the VLE that were unrelated to the learning content. Johnson et al.³⁰ also reported observations that learning sometimes suffered among elementary students using a VLE because they became overengaged with elements of the task itself. These observations taken in combination with the current findings suggest that the complexity of a VLE may influence the relationship between presence and engagement in learning content. Clearly, these findings indicate that this may be an important area for further study.

The experiment we performed had limitations. Although learning content, metaphor, and VE equipment were held constant between the two VLEs, the amount of interactivity, movement, and other factors clearly differed. On one hand, these factors are confounded with mode of learning content delivery; on the other hand, these variables are likely to vary naturally with these types of learning modalities. Additional research should investigate these factors individually.

In all, we believe our results demonstrate that presence does not operate uniformly across VLEs; rather, its effects can differ based on environmental features. We therefore do not expect greater presence to always improve learning outcomes. Furthermore, some VLE aspects, including educational delivery method and perhaps complexity, may influence the impact of presence on engagement with learning content. Attention to these and other aspects of VLEs may be crucial in understanding how to best deliver educational content using VE technology.