Gaming Mindsets: Implicit Theories in Serious Game Learning

Abstract

Individuals' beliefs about the malleability of their abilities may predict their response and outcome in learning from serious games. Individuals with growth mindsets believe their abilities can develop with practice and effort, whereas individuals with fixed mindsets believe their abilities are static and cannot improve. This study uses survey and gameplay server data to examine the implicit theory of intelligence in the context of serious game learning. The findings show that growth mindset players performed better than fixed mindset players, their mistakes did not affect their attention to the game, and they read more learning feedback than fixed mindset players. In addition, growth mindset players were more likely to actively seek difficult challenges, which is often essential to self-directed learning. General mindset measurements and domain-specific measurements were also compared. These findings suggest that players' psychological attributes should be considered when designing and applying serious games.

Introduction

In recent years, many digital games have been designed for educational purposes and professional trainings. These games, whose chief purpose is not pure entertainment, are generally known as serious games.¹ Compared to traditional educational formats such as textbooks, serious games have the advantage of presenting content within simulated contexts. These simulated game environments reduce the cost and risks associated with failing, encouraging learners to take on different roles and experiment with creative solutions.^2–6

It is important to examine how learners respond differently to learning from serious games because just like in any educational format, some learners will learn more effectively from serious games while other will become disadvantaged. Previous studies of serious games have focused on testing design elements that improve learning outcomes. Few studies have examined how learners' psychological attributes may affect learning outcomes.

This study draws on Dweck's implicit theory of intelligence,^7,8 a theory about how people's beliefs about ability and intelligence influence response to failure, behavior, and learning outcomes. The theory has been empirically examined in predicting learning response,^7–9 stereotypes,¹⁰ employee engagement,¹¹ and sports.¹² This current study examines the theory's utility in predicting learning behavior and outcomes in a serious game. The study also compares whether domain-specific measurement of mindset is more predictive than general mindset measurement.

Growth and Fixed Mindsets

Decades of research by Dweck et al.^7,8,13,14 have found that different beliefs (or mindsets) about the malleability of one's ability can result in different learning behavior and outcomes. Two general mindsets were identified. People with fixed mindset (or entity theorists) believe that abilities are fixed, either innate or fully developed in one's early life stages. A person cannot do much to change his or her abilities. By contrast, people with growth mindset (or incremental theorists) believe that abilities can be developed through learning and practice; therefore, if one puts extra effort into learning tasks, one's abilities can grow incrementally through practice.

Previous studies have shown that people with these two mindsets behave differently when facing challenges.^7–9,15 Since people with fixed mindsets believe their abilities are fixed, they regard each challenge as an evaluation of their ability. They are more concerned about how competent they appear than about actually learning. In order to appear competent, people with a fixed mindset tend to seek familiar challenges and avoid challenges where they might fail. They also avoid showing effort because they believe that success is based on intelligence, and smart people should succeed without much effort. On the other hand, growth mindset people perceive challenges as learning opportunities. They seek hard challenges and regard an easy task as boring because it does not help them learn. We hypothesize that growth mindset players will be more likely to seek challenges in a serious game than fixed mindset players.

People with fixed and growth mindsets also have different responses to failure. Fixed mindset people regard failures as evidence that they are incompetent. Since there is not much they can do to change their abilities, they are more likely to quit or cheat.⁹ By contrast, growth mindset people regard failures as a chance to correct and improve themselves. Having a fixed mindset can result in a debilitating response to failure, particularly in the face of prolonged challenges or setbacks.^8,15 We hypothesize that when fixed mindset players make more mistakes in a serious game, they will lose attention and show more negative affect.

Studies in neural psychology have shown that people with the two mindset orientations respond differently to learning. Mangels et al.¹⁶ used brain imaging to compare how people with fixed or growth mindsets reacted to feedback about failure. The participants answered a large set of trivia questions and then received performance feedback (how well they did) and learning feedback (the correct answers). They found that both groups paid attention to performance feedback, but those with a fixed mindset were fixated on the performance feedback and did not attend to the learning feedback. As a result, on a surprise retest, growth mindset participants improved but fixed mindset participants made the same errors again. Another experiment using electroencephalography (EEG) identified a neural mechanism (Pe amplitude) that mediates the relationship between mindset and post-error performance.¹⁷ Specifically, growth mindset was associated with enhanced Pe amplitude—a brain signal reflecting the conscious attention allocation to mistakes—and improved subsequent performance. We hypothesize that growth mindset participants will devote more attention to learning feedback than fixed mindset participants. We also hypothesize that, on average, growth mindset players will perform better in a serious game than fixed mindset players.

It is important to note that mindsets are domain-specific; a person can simultaneously hold different mindsets in different domains.^8,11 However, previous empirical studies^9–12,18 have used either general or domain-specific measurements without comparing the ability of these measures in predicting behavior.

Research Method

We recruited 233 undergraduate students from two large universities for this study. Extra credit was provided to participants as an incentive. We gave the participants a URL that linked to the research Web site; they could participate in the experiment from their personal computer or public computer lab. This condition simulated assigned serious gameplay where the participant is not pressured by supervisors in the same room. Participants first filled out a pre-experiment questionnaire measuring their previous game experience; their favorite game genres; demographics such as sex, age, race, and education; and two versions of Dweck's implicit theory of intelligence scale¹⁴—the original scale measuring general intelligence and one tailored to implicit gaming ability.

The participants were instructed to play the learning game Do I Have a Right? for at least 10 minutes, but they were allowed to play as long as they desired or quit before 10 minutes. The experiment Web site kept track of the participants' in-game behaviors. After playing the game, participants filled out a post-experiment questionnaire measuring their responses to the game, and including positive and negative affect and likelihood they would play the game if not assigned to do so.

The Game

Do I Have a Right? is a serious game for teaching United States constitutional amendments. The game was designed by former Supreme Court justice Sandra Day O'Connor and educational scholars from Arizona State University, and it was developed by Filament Games. Players play as the owner of a new law firm, and their job is to match clients with various legal problems to lawyers specializing in different amendments. When a player correctly matches a case, he or she earns prestige points, which can later be used to hire more lawyers or buy upgrades. Adding more lawyers allows players to cover more amendments, and buying upgrades can either make the game easier or more challenging by attracting more cases. A failed match results in losing cases, which is the main indicator of failure for the game. Learning can take place when the player reads about the amendment descriptions during gameplay and when they read feedback about correct or incorrect matches between rounds. A complete game session constitutes seven game days (rounds), but players can repeat the game many times to improve their performance.

Because the participants played for unequal time, comparing their gameplay and learning based on total accumulated scores would be conflated with duration of play. Gameplay in Do I Have a Right? progresses through seven virtual “days.” We chose to compare scores and other player behaviors on the fourth game day (approximately 6 minutes of play). Four virtual days was sufficient gameplay for players to have become familiar with how the game works and to adopt a play style, yet short enough to include a majority of study participants. This procedure yielded comparable gameplay data from 149 participants (63%), all of whom had completed at least 4 days of play.

Measurements

Mindsets

We measured two types of mindsets. General mindset was measured using Dweck's four-item implicit theory of intelligence scale.^13,14 The items are 6-point Likert-scale questions such as “you have a certain amount of intelligence, and you can't really do much to change it.” A reliability test for general mindset yielded a Cronbach's alpha of 0.97. Because mindsets are domain specific, we also adapted the four general questions to address gaming abilities specifically. These items are 6-point Likert-scale questions such as “you have a certain amount of gaming abilities, and you can't really do much to change it.” Gaming mindset had a Cronbach's alpha of 0.93.

Both general and gaming mindset scores were inverse coded, so higher scores indicate stronger orientation toward growth mindset. Following Dweck's approach,^13,14 we categorized participants' mindset orientation into three groups: participants with scores from 1 to 3 were coded as having fixed mindsets (n=43); participants with scores from 4 to 6 were coded as having growth mindsets (n=80); and participants between (but not including) 3 to 4 were coded as ambiguous (n=25), and were excluded from our analysis.

Challenge

Since Do I Have a Right? does not explicitly offer players the choice of difficulty, we measured challenge by whether the participants bought advertisements in the game. Advertisements increase the number of clients that players must handle in each round, increasing the challenge.

Attention

Attention was measured in the post-experiment questionnaire, combining two items using a 9-point scale: “I was completely focused on this game” and “My attention was monopolized by this game.” Attention produced a Cronbach's alpha of 0.84.

Negative affect

Negative affect was measured using post-experiment survey responses. We asked three questions using a 9-point scale: “I felt bad while playing,” “I felt sad while playing,” and “I felt uninvolved while playing.” Negative affect produced a Cronbach's alpha of 0.78.

Learning

Learning consisted of two variables: (a) total time spent looking at amendment descriptions, which reflects potential learning during gameplay, and (b) total time spent on feedback screen. After every round, players are presented with detailed feedback explaining every correct or incorrect choice they made in that round. This variable reflects potential learning between rounds, including detailed feedback about player choices.

Performance

Performance was measured using percentage of cases won. This is a calculated by dividing the number of cases won by total cases encountered. Using percentages instead of numbers of cases won allows for equal comparison of performance, even though players may have encountered a different number of cases in 4 days of play.

Findings

Overall, general mindset (M=1.71, SD=0.45) was higher than gaming mindset (M=1.65, SD=0.48). A paired sample t test showed that they were significantly different, t(145)=2.09, p<0.05. More participants believed that general intelligence could be improved through efforts (71.1%, n=96) than believed general intelligence is fixed (28.9%, n=39). Gaming mindset showed similar trends. More participants believed that their gaming abilities could improve from efforts (65%, n=80) than believed gaming abilities were fixed (35%, n=43).

In order to compare how well general mindset and gaming mindset predicted learning behavior and performance, we conducted multiple regression on each of the dependent variables (challenge, attention, negative affect, learning, and performance) using the uncategorized continuous general mindset and gaming mindset scores as independent variables.

General mindset did not predict any of the five dependent variables, but gaming mindset was a significant predictor for challenge (B=0.14, p<0.05), negative affect (B=−0.38, p<0.01), and performance (B=0.03, p<0.05). These results support the use of domain-specific mindset measurements over general mindset measurements. Therefore, we used gaming mindset for the subsequent analyses.

Our first hypothesis predicted that growth gaming mindset participants would seek more challenge than fixed gaming mindset participants. The data support this hypothesis. An independent sample t test showed that there was a significant difference in the number of advertisements bought (which increases difficulty) between growth and fixed mindset participants, t(121)=−2.28, p<0.05. Growth mindset participants bought more advertisements (M=0.75, SD=0.94) than fixed mindset participants (M=0.37, SD=0.76). Although this does not indicate that fixed mindset participants avoided challenges, it does suggest that they are less likely to actively seek difficult challenges.

Our second hypothesis predicted that for fixed mindset participants, the number of mistakes would predict loss of attention. The data were consistent with this hypothesis. Hierarchical regression was conducted to test this hypothesis. The number of cases attempted was entered into the first block as a control variable, and the number of cases lost was entered into the second block. For fixed mindset participants, the overall model including first- and second-order predictors was significant, F(2, 36)=4.13, p<0.05, R²=0.19. The number of cases lost (mistakes) was a significant predictor of attention (B=−0.58, p<0.01), adding 18.6% of variance explained after controlling for the total number of cases tried. In order to examine whether this effect was specific to fixed mindset participants, we conducted the same analysis on growth mindset participants. The overall model was not significant, F(2, 71)=0.48. These results suggest that fixed mindset players seemed to lose attention to the game when they make mistakes, whereas growth mindset players' attention was not affected by their mistakes. However, since our attention measurement is self-reported after playing the game, it is also possible that because fixed mindset participants treat mistakes as signs of incompetence, they tend to report lower attention to justify their mistakes. In other words, it is not lack of ability but lack of attention that caused the mistakes.

Since the implicit theory of intelligence posits that fixed mindset people regard mistakes as evidence of their incompetence, our third hypothesis predicted that fixed mindset participants would report experiencing higher negative affect when they made more mistakes. The data were not consistent with this hypothesis. Hierarchical regression was conducted to test this hypothesis. The number of cases was entered into the first block as a control variable, and the number of cases lost was entered into the second block. The overall model was not significant for fixed mindset participants, F(2, 36)=0.70, nor for growth mindset participants, F(2, 71)=2.49. The results indicate that the number of mistakes did not predict negative affect among the participants. However, fixed mindset players (M=3.27, SD=2.09) did report significantly more negative affect than growth mindset players (M=2.38, SD=1.21), t(110)=2.86, p<0.01.

Based on the implicit theory of intelligence and previous study,¹⁶ our fourth hypothesis predicted that growth gaming mindset players would focus more on learning than fixed gaming mindset players by reading more amendment descriptions and spending more time on feedback. We tested the two learning behaviors separately, and the data were partially consistent with this hypothesis. An independent sample t test showed that there was no significant difference in learning behaviors during gameplay (time spent reading amendment descriptions), t(121)=−0.82, between growth and fixed mindset participants, but there was a significant difference in time spent reading feedback that included information that would lead to future success (number of headlines read), t(121)=−2.00, p<0.05). Growth mindset players (M=2.58, SD=2.14) read more learning feedback than fixed mindset players (M=1.77, SD=2.13). In order to examine whether this result was conflated by a different number of cases encountered by fixed and growth mindset players, we conducted an independent sample t test to examine whether the two groups encountered a different number of cases. The result showed that there was no significant difference, t(121)=−1.68. These results indicate that there was no difference in learning during gameplay, but a significant difference in learning from feedback. Growth mindset players focused more attention to learning why they failed than fixed mindset players.

Our fifth hypothesis predicted that, on average, players with growth gaming mindset would perform better than players with fixed gaming mindset. The data were consistent with our hypothesis. Independent sample t test results showed that there was a significant difference in percentage of cases won, t(121)=−2.76, p<0.01. Players with a growth gaming mindset (M=0.87, SD=0.15) won 87% of the cases they brought to trial, performing significantly better than players with a fixed gaming mindset (M=0.78, SD=0.20).

Discussion

This study examined whether beliefs about whether one's gaming ability predicts one's learning behavior and performance in a serious game. The results were mostly consistent with our hypotheses. In the context of serious games, good performance in a game should indicate that learning has occurred. We found that growth mindset players performed better than fixed mindset players, their mistakes did not affect their attention to the game, and they read more learning feedback than fixed mindset players. In addition, growth mindset players were more likely to actively seek difficult challenges, which is often essential to self-directed learning.

Our results contribute support to the implicit theory of intelligence in two ways. First, although the theory states that mindsets are domain specific, previous studies have used general measurements regardless of task domain. To our knowledge, no previous studies have compared general measurements to domain-specific measurements. This study showed that people have different levels of belief about general intelligence and gaming abilities. General mindset did not predict the various learning response, whereas the domain-specific gaming mindset was a significant predictor for three of the five responses. These findings suggest that future studies of mindsets should use more domain-specific measurements of mindsets.

Previous studies were designed so that performance and learning feedback was given at the same time. Mangels et al.¹⁶ and Moser et al.¹⁷ both gave concurrent performance (right/wrong) and learning feedback (what the correct answer was). In other words, participants learned whether they had been right or wrong and what the right response should have been at the same time. This current study also found a significant difference between mindsets on attention to learning feedback, even when learning feedback was delayed until the end of the virtual day. It seems that to fixed mindset players, the pain of being wrong persisted and interfered with paying attention to learning feedback even when the learning feedback was separated in time.

Finally, this study extends the theory to the context of serious game learning. As an increasing number of serious games are assigned in school curriculums and corporate trainings, the practical implication of this study is that one's gaming mindset orientation may predict one's response to learning from serious games. However, this result does not imply that people with fixed mindsets will always remain disadvantaged learners. Previous studies have shown that growth mindset can be taught. Reading a short article about the malleability of intelligence can have short-term effects on learning,^9,19 and a continuous intervention about growth mindsets was found to have longitudinal effects on academic performance.¹⁸ Education practitioners could identify and teach fixed mindset individuals to treat serious game-based learning with a growth mindset, which may improve their learning experience. Education practitioners could also work with game designers to incorporate adaptive designs that adjust the game according to users' mindset orientations. This approach could improve the effectiveness of serious games.

Acknowledgments

This research is partially supported by grant #0943064 from the National Science Foundation. The opinions expressed are those of the authors and do not represent Michigan State University, Georgia Institute of Technology, or the National Science Foundation.

Disclosure Statement

No competing financial interests exist.