Evaluating Motivation for the Use of an Electronic Health Record Simulation Game

Abstract

Experiential learning via simulation offers a variety of benefits including reduced risks, repetitive exposure, and mastery of complex processes. How to motivate people to engage in and enjoy playing games is an important concept in the creation of serious games focused on learning new skills. This study sought to determine the motivators that increase users’ pleasurable experience when playing an electronic health record simulation game. To examine how intrinsic and extrinsic motivation affected both engagement and enjoyment, we surveyed students of health professions at one university. Results indicate that while both forms of motivation are significant in increasing engagement and enjoyment, extrinsic motivation such as badges, points, and scoreboards were much more important than internal motivations for our participants. These findings have implications for the development of an electronic health record simulation game.

Introduction

Digital health information maintained in electronic health record (EHR) systems is an asset of significant value to both patients and healthcare organizations that requires governance by well-trained and experienced health information management (HIM) professionals. Students who will become HIM professionals must develop competency and efficiency in a variety of complex processes governing health information, which is the basis for clinical decisions that affect patient health. Health information must be accurate, complete, and available when and where it is needed for use in clinical and business decisions by all health professionals. The terms technology iatrogenesis or e-iatrogenesis refer to errors that occur because of technology introduced into the already complex healthcare delivery system.¹ To minimize these errors, HIM professionals must achieve high levels of competency and efficiency through effective instruction and extensive practice. Issues with health information create substantial risks for patients, providers, and facilities. Furthermore, health information is highly targeted for compromise by perpetrators of medical identity theft as well as those seeking ransom.² Governance of health information addresses not only the risks to patient health, but also the risks to patient privacy. The challenges associated with health information governance are evolving rapidly and result in the need for new tools and methods to learn and practice these skills in a safe, low-risk environment, such as that provided by simulation.

This article presents the results of a study examining health profession students’ perceptions of game design elements and how these elements can increase the user's enjoyment when playing games. This study is based on the application of simulation and gamification techniques to enhance the experience of health profession students learning to use EHR systems. Inspired by previous experience teaching complex systems and encouraged by the literature about simulation in other areas of healthcare, the authors became curious about the effectiveness of a comprehensive simulation platform specifically designed to enable teaching and repetitive practice of the complex high-risk processes associated with EHR systems and how simulation game design might affect user enjoyment and engagement.

As a step toward understanding the potential of an EHR simulation platform, we conducted a survey of students in our college of health professions to explore game design elements, how these elements motivate participants, and the positive effects on engagement and enjoyment. In the sections that follow, we first provide a review of literature that informed our research question. Next we review motivation and outcomes. After the literature review, we describe our methods and results. We close with a discussion of findings and conclusions drawn from the study.

Literature Review

The current literature on gamification appraises a variety of elements associated with game design.³ Game genres, types, motivations, and other game design characteristics are important in understanding specific game preferences and motivations that might contribute to the design of an EHR simulation game. The purpose of the research was to discover what game design elements motivated HIM students from both an intrinsic and extrinsic perspective with the ultimate goal of making a simulation game engaging and enjoyable.

Gamification

Although the literature includes many examples of simulation and experiential learning in healthcare settings such as anesthesiology,^{4, 5} cardiology,^{6, 7} intensive care,^{8, 9} nursing,^{10, 11, 12} obstetrics,^{13, 14} outpatient healthcare,^{15, 16} and many other disciplines,¹⁷ research on the combination of gamification and EHRs is sparse. A notable exception is that of Mohan et al.,¹⁸ who provided six designs for consideration in development of an EHR simulation. One curriculum development study reported that simulation might be helpful and effective in EHR instruction using a “well-organized chart,” which was defined as documentation of actions reflecting current standards.¹⁹ Additionally, Kuljis et al.²⁰ suggested that simulation in healthcare could benefit from a review of simulation in other settings, such as business manufacturing.

Both game genre and game type should be considered in the design of an EHR simulation game because they allow designers to customize the game, addressing specific learning objectives and taking advantage of analogous games that have already been designed.²¹ Genres such as fantasy, discovery, and serious games need to be taken into account to support and frame the design process. Common categories of game types include shooter, adventure, strategy, puzzle, and sports games. Although some games are played entirely for fun and entertainment, serious games are used as learning platforms, integrating aspects of real-world learning or skill development, and have been created for a variety of purposes, including medical education, retail, service industry, public safety, weight management, surgery, health education, and diabetes management.^{22, 23, 24, 25, 26} An EHR simulation game would be considered a serious game because students would be engaged in learning and skill development while playing to learn in an enjoyable, safe virtual environment.

Unlike serious games, fantasy games provide virtual worlds where many people opt to play, avoiding reality.²⁷ Often, fantasy games are set in imaginary worlds and incorporate fantasy characters, graphics, narratives, and scenarios, providing alternate realities. A large advantage of these imaginary worlds is that users are insulated from repercussions.^{28, 29, 30} Although our literature search did not reveal any previous works relating EHRs to fantasy games, gamifying of learning materials is particularly advantageous in the healthcare field because it facilitates learning while removing the risk of error that the patient and healthcare learner may experience in real-life scenarios. With an EHR, real-life dangers that exist include erroneous patient data, patient identification challenges, security breaches, medication errors, and other situations—all of which can be safely practiced in a simulated environment while building both confidence and expertise. However, it is not only the safe environment that makes gamification appealing: increasing student engagement and enjoyment while learning new skills is also beneficial to learners.

Motivation, Enjoyment, and Engagement

The motivation for playing is crucial to increasing player engagement and enjoyment and has been researched extensively. The extent to which a participant engages in games directly correlates with learning outcomes.³¹ Research suggests that a combination of both intrinsic and extrinsic motivation can be used to engage players.³² Rieber³³ found that when an engaging game is intentionally connected to learning content, the content itself becomes more enjoyable. This enjoyment, in turn, supports internal motivation, which is also referred to as intrinsic motivation.^{34, 35} Game designs that support intrinsic motivation include elements supporting personal achievement, self-determination or drive, team play, social relationships, and networks.^{36, 37, 38}

In contrast to intrinsic motivation, extrinsic motivation uses external rewards to motivate the player.³⁹ Certain game design characteristics are suggested to directly influence extrinsic motivation.⁴⁰ Elements of game design that increase extrinsic motivation include the tracking of points and recognition-based rewards. Examples of recognition-based rewards include badges, scoreboards, leaderboards, progress bars, and level-of-progression scores.⁴¹ Because the goal is to motivate players both intrinsically and extrinsically, extrinsic game elements might be used to initially engage the user with the expectation that the enjoyment of playing would then intrinsically motivate and engage the user over the long term.^{42, 43}

The end result of motivating participants is to engage them in learning and produce greater academic outcomes.⁴⁴ People learn more quickly when they are engaged in content designed to be fun, resulting in deeper understanding, greater skill development, more confidence, and greater achievement.⁴⁵ Engagement is often described as a heightened sense of concentration and can result in great enjoyment in which people lose a sense of time and surroundings. This engagement phenomenon has been termed flow theory, and it characterizes an enjoyable increase in interest and productivity.⁴⁶ Hence engagement is an important aspect of game design.

While engagement is important in gamification, so too is enjoyment.⁴⁷ Often enjoyment is overlooked, particularly in the completion of tedious tasks.⁴⁸ Motivation can increase enjoyment if game design elements are properly used to spontaneously generate student interest.⁴⁹ Self-determination theory proposes that people actively participate when they are supported and are comfortably enjoying the acquisition of new experiences.⁵⁰

According to this conceptualization, gamification can be considered to have three main parts:

1. the implemented motivational strategies,

2. the resulting psychological outcomes, and

3. the final behavioral outcomes.

In a systematic review, Hamari et al.⁵¹ conceptualized a structure for evaluating gamification through a three-part framework: the implemented motivation, the psychological outcome that resulted, and behavioral outcomes. Reviewing the gamification literature, they concluded that gamification leads to increased motivation, engagement, and enjoyment.⁵² Motivated participants are comfortable learning new things, are interested in the topic, and enjoy the learning experience.

Research Questions

We crafted our research questions invoking the motivation outcome framework suggested by Hamari et al.,⁵³ shown in Figure 1. Given the preceding review and the motivation outcome framework, we sought to examine the following questions:

• How do extrinsic motivators affect people's engagement?

• What is the effect of extrinsic game motivators on participant enjoyment?

• What is the relationship between intrinsic motivators and participant engagement?

• How will intrinsic motivation affect a person's enjoyment?

Figure 1.

Motivation Outcome Framework

Methodology

Many researchers have examined intrinsic and extrinsic motivations with the goal of improving performance, participation, engagement, and enjoyment.^{54, 55, 56} Methods of measuring these motivations have varied, and historically researchers exploring intrinsic and extrinsic motivation have created latent variable models and used a survey method. We used a model fashioned after prior research to examine the relationships between perceptions of motivation and how it affects pleasurable use by decomposing motivation into two constructs—intrinsic and extrinsic motivation—while incorporating two constructs of pleasurable use: engagement and enjoyment. Figure 2 details the relationship between intrinsic motivation, extrinsic motivation, engagement, and enjoyment in our research model.

Figure 2.

Motivators of Pleasurable Use

Procedures

We obtained institutional review board permission to operationalize our research design. To test this predictive model, we crafted a survey using the literature previously described and distributed it to students majoring in the College of Health Professions at a major public university in the southern United States. The college currently provides degree programs in clinical laboratory science, communication disorders, HIM, healthcare administration, nursing, physical therapy, radiation therapy, and respiratory therapy. Convenience sampling was used, and an e-mail was sent to students in these majors inviting them to take part in the online survey. Seventy-nine participants completed the survey, which is shown in Appendix A. The Likert-scale survey captured the items of interest related to intrinsic motivation, extrinsic motivation, engagement, and enjoyment. Demographics of the respondents are included in Table 1.

Table 1.

Demographics of Respondents

Characteristic	N	%
Ethnicity
White/Caucasian	44	56%
Hispanic/Latino	17	22%
Black/African American	6	8%
Asian/Pacific Islander	7	9%
Other	5	6%
Gender
Female	64	81%
Male	15	19%
Major
Health information management	31	39%
Nursing	28	35%
Physical therapy	8	10%
Healthcare administration	6	8%
Clinical laboratory science	3	4%
Respiratory care	2	3%
Radiation therapy	1	1%

Analysis

We used partial least squares path modeling (PLS-PM) in RStudio to analyze the data following predictive modeling techniques.⁵⁷ PLS-PM analysis is commonly used in conducting latent variable research and provides a robust way of analyzing survey data.^{58, 59, 60, 61} To validate the psychometric properties of the measures, the average variance extracted, Dillon-Goldstein's rho, Cronbach's alpha, and factor loadings were calculated. Table 2 shows each of these values for this model's constructs.

Table 2.

Results of Data Analysis

Construct	Average Variance Extracted	Dillon-Goldstein's Rho	Cronbach's Alpha
Extrinsic motivation	0.68	0.95	0.93
Intrinsic motivation	0.88	0.97	0.95
Engagement	0.85	0.92	0.82
Enjoyment	0.90	0.96	0.94

Although there is no standard method for calculating statistically acceptable composites, the generally accepted rule is for composite reliability to be greater than 0.7.⁶² In this study, the lowest composite reliability was found for engagement at 0.92, followed by extrinsic motivation at 0.95. Thus, composite reliability as measured by Dillon-Goldstein's rho was greater than 0.9 for all constructs, demonstrating sufficient reliability for all latent variables.

The factor loadings for the latent variables were calculated following the method of Sanchez⁶³ with the use of PLS-PM in RStudio. The factor loadings are provided in Appendix B. Validity of individual items was examined by verifying loadings greater than 0.7 for each construct. All items loaded at values greater than 0.70. Nine indicators loaded greater than 0.90, six loaded between 0.80 and 0.90, and two loaded between 0.80 and 0.70. All items loaded greater “on-factor” than “off-factor,” demonstrating good convergent and discriminant validity.

Results

The structural path model was formulated to test the motivation framework. A bootstrap resampling method produced 500 samples in order to obtain the path coefficients and R² quantities. Statistical significance was then calculated for each path by calculating t-tests. Figure 3 shows the β coefficients and p-values extracted via PLS-PM. The model accounted for a significant portion of the variance in enjoyment (R² = 0.69) and engagement (R² = 0.63).

Figure 3.

Results for Motivators of Pleasurable Use

The coefficient or path value for extrinsic motivation toward engagement was β = 0.64, with p < .01. Extrinsic motivation to enjoyment produced β = 0.66, with p < .01. Intrinsic motivation to engagement was also significant, with β = 0.24 and p < .01. Intrinsic motivation to enjoyment yielded a path value of β = 0.27, with p < .01. Table 3 reports our total effects findings for all participants in the survey, including the sample mean, standard deviation, t-statistic, and p-value.

Table 3.

Total Effects

Relationship	Sample Mean	Standard Deviation	t-statistic	p-value
Extrinsic→Engagement	0.64	0.08	7.87	.01
Extrinsic→Enjoyment	0.66	0.07	8.93	.01
Intrinsic→Engagement	0.24	0.08	2.90	.01
Intrinsic→Enjoyment	0.27	0.07	3.71	.01

The total effects findings indicate that these relationships were significant. We performed a power analysis to determine how many respondents would be required for a medium effect size with a power of .80 at an alpha level of .05, following Cohen.⁶⁴ We found that 67 participants would be required when analyzing two independent variables; thus with 79 valid responses we had adequate power for our study.

Discussion

This study quantitatively explored how intrinsic and extrinsic motivation affect individuals’ engagement and enjoyment when playing games. The four research questions in this work explored how intrinsic and extrinsic motivators increase when an individual is more engaged in and/or enjoys learning while playing a game. The first research question sought to determine whether students are more likely to be engaged in electronic games if extrinsic motivators are available. The students responded that extrinsic motivators would increase their engagement in electronic games.

The second question explored whether individuals would enjoy games more if extrinsic motivators were present. The students indicated that they would enjoy gamification if extrinsic motivators such as badges, scoreboards, and/or other reward systems were available. Thus it is important to include extrinsic motivators in electronic games when these games are used to teach EHR concepts.

The third research question considered whether intrinsic motivation increases engagement in computer games. A person may be intrinsically motivated when the person feels that an opportunity is fun or challenging. Intrinsic motivation occurs even in the absence of external rewards and/or reinforcements. Students felt that being internally motivated would increase their engagement in games.

Finally, the fourth research question asked if intrinsic motivation affected a person's enjoyment of an electronic game. Internal motivation should provide support for individual enjoyment and pleasure when playing. This form of motivation drives the individual at a personal level.

Among our population of respondents, the extrinsic motivation scores (.66 and .64) were more than double the intrinsic motivation scores (.24 and .27) for both engagement and enjoyment. While the results for intrinsic motivators were not as high as those for extrinsic motivators, students still indicated that intrinsic motivators influenced whether they enjoyed games and would be engaged in playing games that had a learning element. To ensure that individuals use game-based learning tools, one must ensure that the incentives to learn are fun as well as challenging. These results provide interesting insight into students’ perceptions of whether gamification can increase their engagement in and enjoyment of learning EHR systems.

Anecdotally, we also asked our respondents why they play games. Forty-five reported playing games out of boredom; 40, for the mental challenge; 19, for social interaction; 18, for educational purposes; 2, for physical challenges; and 10, for other reasons. We also asked participants about the types of games they play. In order of importance, the responses were puzzle, strategy, adventure, educational, multiplayer, simulation, none, role-playing, shooter, athletic, platform, other, and serious games. Appendix C provides the counts by game type.

Conclusion

Teaching students to use EHR systems is a challenging task because the systems are large and complex, requiring significant time to learn and practice. Although people can be trained to use these systems, often this training lacks authenticity because it is presented in lectures or case studies and the audience has little incentive to engage in learning the system. However, when gamification is used to teach systems, learners may be more inclined to engage with the technology for longer periods of time. Well-designed gamification motivates learners, increasing their enjoyment and making learning by doing fun. The individuals who responded to this survey were more extrinsically motivated, meaning that adding features such as badges, scoreboards, and/or other reward systems is more important than ensuring that the game is simply challenging or fun. This finding supports the view that gamification of EHR system learning would be beneficial to our target audience.

Limitations

This paper has several limitations. First, this study used students from a single department within a single college of a university. However, because the study was focused on HIM professionals and the survey was completed by HIM students near completion of their program of study, these students are likely to be a good representation of early-career HIM professionals. Early-career HIM professionals often hold jobs requiring them to engage with EHR systems.

Second, this paper proposes implementation of an EHR simulation game; however, we question whether the results would differ after respondents played the game. These results were based on responses to a survey. We question whether the results would be similar in a different type of study, such as an experiment with an actual EHR simulation game.

Third, other factors may affect a person's gaming engagement and enjoyment. The chosen motivations do not represent all factors affecting the dependent variables, and other, more important effects remain undetermined. This study was therefore limited by the selection of the included variables.

Future Research

Although this study examined whether both intrinsic and extrinsic motivation influence one's engagement and enjoyment in a gamified learning environment, other antecedents such as self-efficacy or voluntariness should be explored as well. Future research should explore whether results would differ on the basis of gender and/or ethnicity. Additionally, the current model explores whether intrinsic and extrinsic motivators are both antecedents of enjoyment and engagement; however, it is possible that extrinsic motivators could actually be antecedents to intrinsic motivators. Finally, this research might benefit from an extension of the research model to include other behavioral outcomes, such as intention to use a system and technology acceptance.

Appendix A

Survey Questions

ENG1	I would be interested in learning a skill by playing computer games.
ENG2	I would be more productive if my work were made more game-like.
ENJ1	Gamification would make electronic health record systems more fun.
ENJ2	Gamification would make electronic health record systems more interesting.
ENJ3	I would be comfortable playing an electronic health record game.
EXT1	A computer game should contain numerous levels or areas that can be explored.
EXT2	If small rewards were given for gaining a certain amount of points, I would return to a computer game more often.
EXT3	A computer game can be made more fun by adding challenges.
EXT4	I prefer playing computer games with strong communities that have social components.
EXT5	When playing a computer game, I would enjoy being awarded expert user status.
EXT6	I would use a computer game more often if I were awarded points for performing different tasks.
EXT7	When playing a computer game, I would enjoy collectable online badges.
EXT8	In a game-based electronic health record, I would enjoy an accumulation of points for accomplishing objectives.
EXT9	In a game-based electronic health record, I would enjoy the ranking of players in a game.
EXT10	In a game-based electronic health record, I would enjoy the tracking of player statistics.
EXT11	In a game-based electronic health record, I would enjoy the ability to play the game with others.
EXT12	In a game-based electronic health record, I would enjoy the use of graphics to indicate levels of completion.
EXT13	In a game-based electronic health record, I would enjoy badges awarded as recognition for accomplishments in a game.
INT1	I welcome the introduction of new technology in my studies.
INT2	I consider myself to be “open” to new practices that are introduced in my studies.
INT3R	I would rather new technology not be introduced in my studies.
INT4R	I am generally resistant when new ways of working are introduced in my studies.
INT5	I am willing to learn new skills to take advantage of new technology that is introduced in my studies.
INT6	I look forward to the advantages brought by new practices that are introduced in my studies.

Appendix B

Factor Loadings

Item	Extrinsic Motivation	Intrinsic Motivation	Engagement	Enjoyment
1 EXT1	0.88	0.54	0.79	0.76
1 EXT2	0.79	0.43	0.70	0.64
1 EXT3	0.85	0.43	0.60	0.63
1 EXT4	0.80	0.31	0.55	0.51
1 EXT5	0.77	0.32	0.53	0.54
1 EXT6	0.85	0.43	0.60	0.72
1 EXT7	0.84	0.50	0.64	0.74
1 EXT8	0.83	0.36	0.58	0.69
2 INT1	0.48	0.94	0.58	0.56
2 INT2	0.55	0.96	0.56	0.61
2 INT3	0.45	0.93	0.47	0.56
2 INT4	0.43	0.92	0.50	0.55
3 ENG1	0.74	0.56	0.93	0.71
3 ENG2	0.66	0.47	0.91	0.73
4 ENJ1	0.78	0.56	0.77	0.96
4 ENJ2	0.77	0.58	0.76	0.96
4 ENJ3	0.73	0.60	0.68	0.92

Appendix C

Type of Games Played

Game Type	Number	Percentage
Puzzle	36	16%
Strategy	35	15%
Adventure	27	12%
Educational	24	11%
Multiplayer	22	10%
Simulation	15	7%
None	14	6%
Role-playing	14	6%
Shooter	14	6%
Athletic	9	4%
Platform	6	3%
Other	6	3%
Serious	5	2%