Testing the Effects of Educational Strategies on Comprehension of a Genomic Concept Using Virtual Reality Technology

Abstract

Objective

Applying genetic susceptibility information to improve health will likely require educating patients about abstract concepts, for which there is little existing research. This experimental study examined the effect of learning mode on comprehension of a genomic concept.

Methods

156 individuals aged 18–40 without specialized knowledge were randomly assigned to either a virtual reality active learning or didactic learning condition. The outcome was comprehension (recall, transfer, mental models).

Results

Change in recall was greater for didactic learning than active learning (p<0.001). Mean transfer and change in mental models were also higher for didactic learning (p<0.0001 and p<0.05, respectively). Believability was higher for didactic learning (p<0.05), while ratings for motivation (p<0.05), interest (p<0.0001), and enjoyment (p<0.0001) were higher for active learning, but these variables did not mediate the association between learning mode and comprehension.

Conclusion

These results show that learning mode affects comprehension, but additional research is needed regarding how and in what contexts different approaches are best for educating patients about abstract concepts.

Practice implications

Didactic, interpersonal health education approaches may be more effective than interactive games in educating patients about abstract, unfamiliar concepts. These findings indicate the importance of traditional health education approaches in emerging areas like genomics.

1. Introduction

1.1. Patient education in genomics

Genetic research is making it increasingly possible to provide patients with information about inherited susceptibilities to common diseases [1]. The challenges involved in educating patients about such genomic information are significant, however. For one, applying genomic information to improving health will likely require patients to have some understanding of the complex, multifactorial nature of common diseases and knowledge of related scientific concepts.

Communication about genomics can therefore place substantial information demands on patients, particularly those without related training or experience [2], and those who have limited genetic-related skills and knowledge (“genetic literacy”). About 38% of U.S. adults have limited health literacy [3], and the proportion of adults with limited genetic literacy might well be higher. In particular, levels of conceptual knowledge (i.e., knowledge of background concepts required to understand and use health information [4]) may be insufficient to understand genetic information for many individuals. Although the general public is reasonably aware of genetic risk factors for multifactorial diseases, awareness ranges greatly across different conditions [5], and individuals may not have conceptual understanding of terms like “genes” or “DNA.”[6, 7].

The development of educational strategies to improve conceptual knowledge is therefore important to providing patients with the foundation necessary to understand and apply genomic information. Public understanding of how environmental and behavioral factors impact the effects of genes on health will likely be a critical part of future disease prevention efforts [8]. Prior research provides limited guidance for developing strategies to improve knowledge of the concept that genetic and environmental factors interact to affect common disease risk, and this has been highlighted as an important future direction for communication research [9]. Existing genetic education and communication studies have generally focused on the specialized genetic counseling encounter or the communication of genetic test results within high-risk families [10, 11]. The purpose of this experimental study was to compare two strategies based on different educational approaches to teach lay individuals about the genomic concept of gene-environment interactions.

1.2. Theoretical framework

Theory and empirical research in pedagogy and science education suggest that learning mode (the way in which information is presented to learners) has a critical influence on learning outcomes. These literatures indicate that the traditional mode of didactic learning through listening or reading might not be optimal for education about abstract scientific concepts, because developing an understanding of these concepts often requires building mental models of unfamiliar and intangible objects [12]. Active learning, in which learners are asked to construct their own knowledge through self-driven, interactive activities, has been suggested as a better educational approach for building such mental models [12, 13]. These interactive approaches are thought to promote deep cognitive processes, resulting in active construction of new knowledge [14]. However, little research has examined active learning approaches for educating adults about abstract scientific concepts. Some related research in medical and college science education has shown greater increases in comprehension with active learning compared to more traditional learning approaches [15, 16], but other studies have found no differences [17, 18].

In addition, the mediating mechanisms underlying the proposed effects of learning mode on conceptual understanding are not understood. The limited research in this area has generally not been theoretically based, and potential mediating variables have not been operationalized and examined systematically. Therefore, we chose to explore possible mediating variables suggested by the Elaboration Likelihood Model (ELM), a dual process theory of cognition which distinguishes between central and peripheral routes of information processing [19]. Use of deeper (i.e., central) processing is desired for education because it results in cognitions and attitudes that are more stable over time. Individuals who are motivated to think about and have the ability to process information use the central route [19].

For this study, we therefore identified possible mediating variables related to increased motivation, greater ability, and deeper processing. These variables included elaboration (i.e., thinking deeply about the information), and motivation to engage in the learning activity, as well as factors related to motivation: attention to the information, involvement (i.e., perceived personal relevance of information) and interest in the information [19, 20]. We also identified enjoyment of the activity and believability of the information as variables possibly related to motivation, and perceived difficulty as related to individuals’ ability to process the information.

1.3. Virtual reality technology

To test the effects of learning mode, we used a immersive virtual reality platform. This technology immerses users in a three-dimensional digital environment created via a combination of graphics and scripting software. Wearing an interface, users can be exposed to and act within virtual environments [21]. Immersive virtual reality technology holds promise as an experimental setting for social and behavioral research in genomics [22], as virtual worlds reduce the trade-off between internal and external validity in experiments [23].

For this study, we developed virtual worlds for both active learning and didactic learning approaches. We hypothesized that comprehension of the gene-environment concept would be higher for active learning than didactic learning, and that this association would be mediated by elaboration, motivation, attention, involvement, interest, believability, enjoyment, and perceived difficulty, as described above. In addition, we explored whether education and having a scientific occupation would modify the hypothesized association, due to differences in learning skills and prior conceptual knowledge.

2. Methods

2.1. Study design

This study employed a two group, between-subjects, repeated measures randomized design to test which learning mode (active versus didactic) best conveyed the concept of gene-environment interactions.

2.2. Sample

We enrolled 165 participants who responded to advertisements placed through a clinical research volunteer program; they received $40 for participation. Inclusion criteria included being age 18–40, and not working in genetics or having taken a human genetics course within the past five years. Individuals with self-reported epilepsy diagnoses, vision or hearing problems, third-trimester pregnancy, and vestibular disorders were excluded. Nine participants were excluded from analysis due to incomplete data; the final sample size was 156.

2.3. Study procedure

Participants wore a head mounted display (HMD) as their interface. Their head and body movements were tracked using optical and inertial tracking systems to render the appropriate scene in real time. Visual scenes were rendered stereoscopically, producing three-dimensional “virtual worlds.”.

Each participant completed one experimental session lasting approximately one hour. After finishing the consent process, participants completed a computer-administered pre-test questionnaire. They then listened to a standard audio introduction, after which they put on the HMD and had a short practice session with the equipment. They then completed the tasks in the active or didactic virtual learning worlds, followed by a post-test computer-administered questionnaire. Participants spent similar amounts of time in the two worlds (active learning: mean=11.1 minutes, SD = 2.4 minutes; didactic learning: mean=11.8 minutes, SD = 0.5 minutes). Finally, participants were debriefed. The Institutional Review Board at the National Human Genome Research Institute approved this study.

2.4. Educational interventions

The development of the virtual worlds used here has previously been detailed [24]. In brief, both worlds were structured around a set of five questions presented in the same order. In the active learning world, participants were presented with the questions and asked to determine the answers for themselves; they had to answer correctly before proceeding. In the didactic learning world, participants were told the answers in a lecture. Both worlds concluded with a review of all correct answers.

The worlds also had a common set of learning objectives, on which the comprehension measures described below were based. The worlds referred to a hypothetical disease (“gallbladder hyperposia”) so that participants would not have strong prior beliefs about the disease and to allow the description of a gene-environment interaction with greater detail than would currently be possible for an actual disease. Gallbladder hyperposia was described as an adult-onset, chronic disease with genetic and behavioral (exercise, fat consumption) risk factors. In the hypothetical gene-environment interaction, exercise had a greater protective effect for individuals at increased genetic risk than for those at lower genetic risk.

The virtual worlds were both based upon an elevator metaphor, in which a virtual elevator was controlled by rows of buttons representing possible levels of genetic and behavioral risk factors. Selecting buttons representing different combinations of risk factors caused vertical movement of the elevator. The floor on which the elevator stopped represented the level of disease risk based on that risk factor combination. In addition, when the elevator stopped on a given floor, the doors opened to show 10 virtual people in the lobby, a certain number of which would then enter a “hyperposia” clinic as another representation of that level of disease risk.

In the active learning world, participants could select buttons representing different levels of risk factors themselves, allowing them to experience the effects of these choices on disease risk. In the didactic learning world, participants listened to a lecture given by a virtual health educator describing how different combinations of factors affect disease risk, illustrated by screenshots taken from the active learning world.

2.5. Measures

2.5.1. Dependent variables

We assessed comprehension using three measures:

Recall assessed how much presented information participants remembered (e.g., “Exercising will lower someone’s chance of getting gallbladder hyperposia”). Fourteen agree/disagree items were assessed at pretest and post-test; number of correct answers was summed.

Transfer assessed whether participants could apply what they had learned about the gene-environment interaction to new disease contexts, assuming that the interaction worked in the same way (e.g., “Exercising will lower the chance of getting heart disease more for someone who has a risk version of a gene for heart disease than someone who does not have a risk version of that gene.”) Because participants were directed to answer the six agree/disagree items based on what they had learned about this type of gene-environment interaction, transfer was assessed only at post-test. Number of correct answers was summed.

For mental models, participants responded to two questions. For the first question (“What are some factors that affect your chance of getting a disease?”), responses were scored from 0–2 based on how many risk factor categories (i.e., genes/family history, behavior) were mentioned. For the second question (“How might those factors work together to affect your chance of getting a disease?”), responses were scored on a 0–3 point scale based on how well they matched the concept of an interaction between factors in affecting disease risk. Possible total scores ranged from 0–5. For example, the following responses received a total score of 1: “Having the disease inhearited [sic]. By me not having knowledge of the disease”, while this pair received a 5: “Having a risk version of the gene; Environmental factors; Behaviors. If you have the risk version of the gene that can cause disease, then behavior is more important in determining whether you will get the disease than for someone who doesn’t have the risk version of the gene.” Two independent coders discussed any discrepancies prior to analysis. Because our interest was change in mental models in response to the learning activities, we calculated change in scores from pre- to post-test.

2.5.2. Hypothesized mediating variables

Hypothesized mediating variables were assessed at post-test only.

Elaboration was assessed using the average of three items (e.g., “I found myself thinking about the information in the virtual environment”) answered on seven-point response scales (α =0.74)[25].

Motivation was assessed with the item “If you had a chance to use a program like this again, how eager would you be to do so?”, answered on a seven-point response scale [26].

Attention was assessed with the item “How much attention did you pay to the information in the virtual environment” answered on a seven-point response scale [26].

Involvement items were based on Roser (1990) [27]. The three items (e.g., “Is information about how genes affect disease risk important to you?”) were answered on three-point response scales and averaged (α=0.74).

Interest and perceived difficulty were each assessed using two seven-point Likert scale items (e.g., “How interesting was this information?” and “How difficult was the information?”, respectively) averaged into scales [26]; α=0.81 for interest and 0.82 for perceived difficulty.

Enjoyment of the world was measured using three seven-point Likert scale items (e.g., “I would have liked the experience to continue.”) [28], which were averaged (α=0.85).

Virtual world ratings: We asked participants to rate believability on a seven-point Likert scale (“How believable did you find the information presented in the virtual environment?”), and asked what they liked and disliked about the worlds (e.g., “What were one or two things that you liked about the virtual environment?”).

2.5.3. Sociodemographic variables

At post-test, we assessed age, race/ethnicity, occupation, educational attainment, and whether they had friends or family members who had been diagnosed with gallbladder problems or a genetic disease. Occupations were classified as “scientific” (e.g., health care provider, lab scientist) and “non-scientific” (e.g., law enforcement, accounting).

2.6. Analysis

Data were analyzed using SAS Version 8 for Windows (Cary, NC). Descriptive statistics were examined for all variables. We used one-way ANOVA tests to examine differences by learning mode in change in recall, transfer, change in mental models, and the hypothesized mediating variables. Statistical significance was assessed as p<0.05.

In addition, we built multiple linear regression models to examine mediation and effect modification of tested associations. First, we built two models with the following dependent variables: post-test recall score (controlling for pre-test recall score) and transfer score. We did not build models with mental models score as a dependent variable due to limited variability in that measure. We employed forward checking and backward elimination methods to determine which covariates to include in the final models [29]. We used a p<0.20 criterion for inclusion of covariates in the models, since a p<0.05 criterion can result in omissions of important confounders with weaker relationships to the dependent variable but stronger relationships to the independent variable [30,31]. We tested potential mediators using the approach of Baron and Kenny (1986) [32]. We also tested whether educational attainment and occupation (scientific vs. non-scientific) modified the associations using interaction terms.

3. Results

As shown in Table 1, about half (47%) of participants were female and aged 25 or younger (42%), and the majority were white (67%). Sixty-seven percent had a college degree or higher; 39% had a household income of less than $40,000. Just over half (56%) worked in scientific occupations, while the remaining 44% had other occupations or were not working. Seventeen percent reported that a family member or friend had been diagnosed with a gallbladder problem; 20% reported having a family member or friend that had been diagnosed with a genetic disease.

Table 1.

Characteristics of participants (n=156).

Characteristic	N (%)a
Gender
Female	72 (47%)
Male	83 (53%)
Age
18–25	65 (42%)
26–30	40 (26%)
31+	51 (33%)
Race/ethnicity
White	104 (67%)
Hispanic	10 (6%)
Asian/Pacific Islander	15 (10%)
African American/Black	28 (18%)
Other	8 (5%)
Education
High school graduate	16 (10%)
Some college	36 (23%)
College graduate	58 (37%)
Post graduate degree	46 (30%)
Income
Less than $40,000	59 (39%)
$40,000-$80,000	47 (31%)
$80,000 and higher	47 (31%)
Occupation
Scientific (lab scientist, health care provider)	86 (56%)
Non-scientific (e.g., law enforcement, administrative)	70 (44%)
Family member or friend diagnosed with gallbladder problem
Yes	26 (17%)
No/Don’t know	130 (83%)
Family member or friend diagnosed with genetic disease
Yes	32 (20%)
No/Don’t know	124 (80%)

^aPercentage values may not total 100% due to rounding.

All three comprehension measures were significantly related to learning condition (see Table 2). Mean change in recall between pre- and post-test was significantly higher in the didactic learning condition than the active learning condition (p<0.001). Similarly, mean transfer score at post-test was significantly higher for didactic learning than for active learning (p<0.0001), as was mean change in mental models score (p<0.05).

Table 2.

Comprehension scores across experimental conditions.

	Active learning (n=81)	Didactic learning (n=75)
	Mean (SD)	Mean (SD)
Variable
Recall score
Pre-test	7.3 (1.9)	7.6 (2.0)
Post-test	8.3 (1.5)	9.9 (1.6)
Change between pre-test and post-test**	0.96 (1.8)	2.3 (2.6)
Transfer score at post-test***	2.6 (0.70)	3.0 (1.2)
Mental model score
Pre-test	3.0 (1.1)	2.9 (1.1)
Post-test	3.4 (1.1)	3.7 (0.77)
Change between pre-test and post-test*	0.41 (1.1)	0.81 (1.0)

SD = standard deviation

^*p<0.05

^**p<0.001

^***p<0.0001

As shown in Table 3, among hypothesized mediating variables, mean believability was rated significantly higher for didactic learning than active learning (p<0.05). In contrast, mean ratings for motivation (p<0.05), interest (p<0.0001), and enjoyment (p<0.0001) were all significantly higher for active learning than didactic learning. Ratings for elaboration, attention, involvement, and difficulty did not differ significantly by learning mode.

Table 3.

Ratings for hypothesized mediating variables across conditions.

	Active learning (n=81)	Didactic learning (n=75)
	Mean (SD)	Mean (SD)
Variable
Elaboration	5.2 (1.3)	5.1 (1.2)
Motivation*	5.6 (1.5)	5.1 (1.4)
Attention	5.9 (1.0)	5.8 (1.0)
Involvement	2.8 (0.4)	2.7 (0.4)
Interest**	5.4 (1.3)	4.3 (1.3)
Perceived difficulty	2.6 (1.2)	2.5 (1.0)
Enjoyment of the world**	5.6 (1.2)	4.8 (1.2)
Believability*	5.6 (1.2)	6.0 (1.0)

^*p<0.05

^**p<0.0001

In multivariate analyses, learning condition was a significant predictor of post-test recall score (p<0.0001), controlling for pre-test recall score, occupation, age, race/ethnicity, and having a family member or friend with gallbladder problems (see Table 4). Learning condition was also a significant predictor of transfer score at post-test in a multivariate model (p<0.01), controlling for condition, age, race/ethnicity, gender, and having a family member or friend with gallbladder problems (see Table 5). Occupation did not modify the associations between learning condition and recall or transfer (p=0.22 and p=0.71, respectively), nor did educational attainment (p=0.81 and p=0.38, respectively). Education was a significant predictor of change in recall, however; individuals with greater educational attainment had higher average recall scores than did those with lower educational attainment at post-test in the multivariate model (p<0.01). None of the potential mediating variables we tested were shown to mediate these associations.

Table 4.

Predictors of recall score at post-test in a multivariate linear regression model (n=156).

Parameter	Beta coefficient	p-value
Condition*	−1.48	<0.0001
Recall score at pre-test	0.11	0.062
Education
Did not complete college	−0.58	0.045
College degree or higher	0.00
Occupation
Scientific	0.39	0.11
Non-scientific	0.00
Race/ethnicity
Black, non-Hispanic	−1.05	0.002
Other	−0.27	0.37
White, non-Hispanic	0.00
Family member or friend with gallbladder problems
No	0.51	0.11
Yes	0.00
Age	−0.042	0.08

^*Active learning compared to didactic learning

Table 5.

Predictors of transfer score at post-test in a multivariate linear regression model (n=155).

Parameter	Beta coefficient	p-value
Condition*	−0.40	0.012
Race/ethnicity
Black, non-Hispanic	−0.43	0.044
Other	−0.37	0.066
White, non-Hispanic	0.00
Family member or friend with gallbladder problems
No	0.32	0.13
Yes	0.00
Age	0.030	0.025
Male gender	0.20	0.19

^*Active learning compared to didactic learning

4. Discussion and conclusions

4.1. Discussion

The purpose of this study was to examine the effect of learning mode on comprehension of the genomic concept that genetic and environmental factors interact to affect risk of common disease. Comprehension improved with both active learning and didactic learning approaches. However, contrary to our hypothesis, comprehension was significantly higher with didactic learning than active learning. We observed this effect for three measures of comprehension, both closed-ended and open-ended, suggesting that the result was not a measurement artifact. We did find that, as hypothesized, motivation, interest, and enjoyment were significantly more highly rated for active learning than for didactic learning, while, contrary to our hypothesis, believability ratings were significantly higher for didactic learning.

No prior research has systematically examined the effects of learning mode on genomic conceptual knowledge, although the small pilot study we conducted prior to the present experiment with a different sample of 40 undergraduates also suggested that comprehension increased more with didactic learning [24]. Despite the general endorsement of active learning approaches in existing literatures, some researchers have called for a reassessment of the effectiveness of this approach and have highlighted the importance of research to understand how it actually affects learning outcomes [33]. Active learning has sometimes [15, 16], but not always [17, 18], been found to lead to greater increases in knowledge. The lack of consistent operationalizations of “active learning” across studies also makes it difficult to compare results and identify the effective components of active learning [16].

Some researchers have suggested that the effectiveness of active learning compared to didactic learning might depend on factors including learners’ affect or higher-level thinking skills [34]. Others in science education have proposed that a hybrid between didactic and active learning might produce the best outcomes [35]. Level of learners’ prior knowledge has been found to influence the effectiveness of interactive e-learning environments [14], and individuals’ pre-existing cognitive representations of an illness threat can impact their responses to new genetic information [36]. It is possible, therefore, that some didactic component to increase prior conceptual knowledge is required to optimize learning in an otherwise active learning approach. Given that didactic learning is the dominant paradigm in fields such as patient education and genetic counseling, an interesting area for further investigation is whether adding active learning components to standard approaches will improve their effectiveness, and whether this differs by patients’ prior knowledge, affect or higher-level thinking skills.

Further investigation is also needed to examine the mediating pathways by which different patient education approaches affect learning outcomes. Interestingly, in some prior studies that did not observe differences between active and didactic learning on comprehension, participating students were shown to rate active learning as more enjoyable or more engaging [17, 37, 38]. These findings, together with our results, at least suggest that participants might be more engaged with or enjoy active learning approaches more because of distracting features that do not improve learning, or their enjoyment of the approach may itself distract from learning. Closer examination of what aspects of active learning approaches participants attend to is an area for further investigation.

Another possibility is that variables such as motivation, interest, and enjoyment may not mediate the association between learning mode and comprehension. Other adult learning or information processing theories might provide a better basis with which to select possible mediators. It is possible, for example, that individuals might value didactic learning more than active learning because it is more familiar [37], leading to better learning outcomes. Participants’ perceptions of the interactivity of learning approaches, and their ability to control the learning environment, might also affect comprehension. In investigating possible mechanisms, it will also be important to examine the stability of learning over time. The ELM suggests that central processing results in more stable learning. It is therefore possible that some mediating variables examined here might predict long-term learning outcomes rather than immediate outcomes, while didactic approaches might be more effective for short-term learning.

One of the exploratory aims of this study was to examine whether educational attainment or having a scientific occupation modified the relationship between learning mode and comprehension. Previous health education research has shown differential effects of information presentation across population subgroups, for example, by level of health literacy or student ability [39, 40]. In addition, some existing research has shown the difficulty of communicating genomic risk information to audiences with limited education or literacy [41, 42]. However, the findings from the present study showed that didactic learning was more effective than active learning in increasing comprehension across educational and occupational categories. There were limitations to the variability in these variables, however, and future research could examine the effects of learning mode in more diverse populations.

It will also be important to investigate the effects of learning mode on comprehension using real-world patient education settings in order to investigate whether the associations observed here are the same and to enhance the generalizability of findings. Some prior research has found no difference in learning between virtual reality and desktop computer conditions [43], while other studies have found that learning improves in virtual reality [40, 44]. Therefore, the effects of active learning versus didactic learning via desktop approaches could be examined in the context of patient waiting rooms in clinic settings. Alternatively, the effects of an actual health educator delivering educational messages could be compared to those of a desktop active learning game for groups of patients. It may be that active learning does increase motivation in “real world” educational settings, but that the effect is not observed in a laboratory setting in which all participants have an artificially heightened motivational state.

The limitations of this study should be considered in interpreting the results. The participants were self-referred volunteers and may have had greater interest than an unselected sample. In addition, participants were relatively young; information about risk of an adult-onset disease might be more salient for older adults, potentially affecting the generalizability of the results. The measures were self-reported, and therefore subjective. The measure of transfer was limited to whether participants could apply their understanding of this particular type of gene-environment interaction across disease contexts, and might not reflect overall understanding of disease causation across different types of genes, behaviors, and diseases. Similarly, individuals’ mental models might vary across different disease contexts, which would not be captured with our measure. Additional research is also needed to examine whether learning predicts subsequent changes in protective health behaviors.

4.2. Conclusion

Existing research provides little guidance for the development of strategies to communicate abstract genomic concepts to patients, and factors such as information presentation are critical to investigate. This study examined the effects of learning mode on comprehension of the concept that genetic and environmental factors interact to affect disease risk. The findings that increases in comprehension were greater with didactic learning, while motivation, interest and enjoyment were rated more highly for active learning, highlight the importance of systematically investigating mechanisms underlying the effects of different learning approaches in patient education. More research in both laboratory and real-world settings is needed to examine the conditions under which different learning modes are most effective for improving conceptual knowledge, and for which populations.

4.3. Practice implications

Developing strategies to improve conceptual knowledge is likely to be critical in assisting patients to understand their genetic susceptibility to common disease and applying this information to improve their health. This study showed that a didactic lecture approach was more effective than an active learning (i.e., interactive, game-like) approach in improving understanding of an abstract genomic concept, at least with respect to short term learning. In designing educational approaches for abstract concepts that are likely to be unfamiliar to patients, some active learning approaches, such as interactive games, might not be as effective as interpersonal didactic approaches, perhaps due to the presence of distracting features or limitations in patients’ related prior knowledge and skills. These findings therefore indicate the continued importance of more traditional health education approaches in emerging areas such as genomics.