Immune Attack players perform better on a test of cellular immunology and self-confidence than their classmates who playa control video game

Introduction

As molecular scientists, we understand the fundamentals of molecular behavior that underlie evolution, infection, physiology and environmental contamination. However, the basic fundamentals of molecular behavior are considered too abstract to teach to grade school or high school students in the context of cellular biology. Analogies are used to describe the functions of cells and their organelles. Concepts such as protein production and respiration are taught from a systemic, large scale perspective, typically without presenting the role of any individual proteins in these cellular processes. The molecular behavior of say ATP and its random diffusion is not taught in context with Mitochondrial function. The Next Generation science teaching standards do not emphasize molecular behavior, and explicitly state that assessment of students should not include any information about individual proteins or a biochemical level of understanding of cellular processes(7).Only students who choose to take an advanced level high school course or who opt into college level biology courses are exposed to the fundamentals of molecular biology.

This lack of detail and lack of exposure through high school to the fundamentals of molecular behavior leave the general public without the basic understanding required to grasp cellular biology or to understand new data. Because this new data often pertains to personal and public health decisions, these concepts are important for non-scientists to understand. Memorizing every step of glycolysis is not necessary. However, a fundamental understanding of how cellular processes are driven by concentrations of substrates, products, allosteric inhibitors; how enzymes, products and substrates diffuse in random directions, that through random interactions specific binding occurs due to very particular aspects of shape, charge and other molecular forces and that processes require a particular enzyme for each step would serve students as a scaffold for a life time of learning.

Brunner’s work on a game (The Ruby Realm) to teach photosynthesis to middle school students demonstrates that trying to teach cellular processes like photosynthesis without presenting the molecular interactions leads to misconceptions. She found that students consistently had trouble understanding that the mass of a plant comes from the sun and air, rather than the dirt, until she introduced the molecules and their interactions in the game. Using analogies and large scale systematic approach (as recommended by most science teaching standards) did not help students comprehend photosynthesis because the large scale characteristics of dirt, air, and plant material are so different. By allowing students to “shoot” carbon dioxide molecules with sunlight and form glucose molecules with them, Brunner was able to teach a deeper understanding of photosynthesis to younger students (REF).

Klymkowsky and colleagues have demonstrated that misconceptions are commonly held by biology undergraduates as they enter college. Additionally, these misconceptions concern fundamental molecular behaviors, such a diffusion and the random motion of molecules and proteins(5). Klymkowsky reported on a multi year effort to address misconceptions in undergraduate biology classes and concluded that these misconceptions are difficult to erase and re-teach (5). Teaching the basis of molecular behavior to younger people can help address this problem.

Jenkinson and McGill have shown that showing greater detail in biochemistry videos achieves a higher level of understanding. Namely, they created four versions of a receptor and its ligand interacting in space. Where typical videos show a ligand and receptor only and place the ligand on a clear path to the receptor, Jenkinson and McGill increased the randomness of the ligands approach and also added water molecules that collide with the ligand. They created four variations of the receptor and ligand video. They found that the more complicated videos were best for evoking a deeper understanding the process of ligand randomly approaching and binding its receptor. Jenkinson and McGill found that this deeper understanding wore off after two weeks, as the groups that saw the more complex videos failed to answer the deeper understanding questions correctly two weeks later (3). Jenkinson and McGill state that complexity may be required to properly teach molecular interactions and that videos with audio and interactive elements may be the more effective.

What is missing is a tool to present the complex, abstract fundamentals of molecular behavior to younger students so that by the time we teach them the specifics of respiration or photosynthesis they can understand these processes deeply. The tool would 1) present a great deal of detail, presenting many types of objects that interact and have numerous traits that dictate their activities, 2) allow for interaction and puzzle solving: allowing students to manipulate, explore and use the objects they find to affect the state of the molecular world. 3) be engaging, fun and interesting to non-scientists so that the general public and younger students will take advantage of the tool 4) attract students back to play again and again over a long period of time so that the deeper understandings do not wear off and so fundamentals used in the tool have an affect on what the students are currently learning in school.

The tool we need has already been created: the video game (2, 9). A video game with a story, set in a complex molecular world, in which the players not only watch, but also interact with and listen to in-game characters may well be the best way to impart understanding of complex interactions among molecular entities. Complex games with multiple ways to achieve success often bring players back to re-play, often over many years time. A game set in a real, complex world of molecules and cells may be an ideal way to introduce molecular behavior.

Successful commercial video games, that keep players coming back to replay, have a learning curve: a period of time and practice is required of the player to learn the game mechanism and the controls. A game complex enough to engage players in a story, with the ability to manipulate and experiment with objects will have a lengthy learning curve, as opposed to a casual vocabulary guessing game. Therefore, we are interested in knowing whether a complicated, third person shooter that involves controlling a ship in 3D space as well as shooting targets will have a learning curve that inhibits learning of the terms and concepts presented. Additionally, we are interested in knowing whether this learning curve inhibits players’ confidence with the material.

Commercially successful video games do not have a list of multiple choice questions to answer or a vocabulary test. However, each level requires learning. Often some vocabulary is required to find objects and places on a map, or to choose weapons or to read tips about how to kill aliens. There are many potential ways that learning may occur in a video game. Therefore, we are interested in knowing whether we can demonstrate any retention of terms and concepts when the game itself does not ask any explicit questions of the player. Will players learn vocabulary that they are not forced to recall in the game? Will players learn about processes? Will they remember attributes of objects that they are not required to interact with? Will they be able to extrapolate what they learn to the real world? Will the game graphics, vocabulary, and situations prepare them for working with real materials on the subject? These questions are addressed by our evaluation of Immune Attack, and our data is presented in Results and Discussion.

There is precedent that concepts can be learned in a story oriented, commercially successful game that is not designed to teach explicitly. Civilization is designed to be an engaging experience, not to teach facts about history. However, Kurt Squire demonstrated that middle school students gained a valuable perspective on the situations, geographies, issues and technology innovations that lie at the heart of many events we learn about in history. In summary, Civilization players who have defended a large empire on a mountain from many smaller invading groups can change their strategy when the start their city on a low plains area. (10). In other words, the players have learned about how armies and geography interact in general. This general understanding of the complex interactions should provide the player with a scaffold for a deeper understanding of history. This type of general understanding of the interactions between molecules and cells is what we propose that a video game can provide. This understanding of molecular behavior is what we think will foster a deeper understanding of health and biology related curriculum.

Immune Attack is a video game that requires players to activate specific proteins to cause specific behaviors of various white blood cells to win seven game levels. Immune Attack is a third person shooter styled video game. Players fly a microbot and a nanobot through veins, through connective tissue and over the surface of white blood cells, while receiving instructions to “shoot” various proteins at appropriate times to solve various failings of a patient’s immune system. Shooting proteins with an “EM emitter” causes the proteins to be come active and perform their task.

Immune Attack is designed so that the action of the level is blended well with the science. Through a simple game mechanism (point and shoot) Immune Attack is able to present a wide range of molecular situations to players without introducing a new game mechanism. And the mechanism is reasonably well matched with the meaning of the game.

Proteins and cells can be manipulated in Immune Attack, they have roughly accurate sizes, shapes and clear roles to play in the action and drama of each level. In Immune Attack, molecules and cells are not just words we need to memorize, they are tools we need to use. Therefore, objects and concepts have scaffolding upon which to learn them. This should allow Immune Attack to present more detail and more abstract concepts to younger audiences. We also expect, that this use of these objects in the game will foster students’ abilities to recall their names and functions after playing.

Immune Attack is unique among learning games because 1) it presents advanced, abstract and fundamental concepts in cellular and molecular biology and 2) does not explicitly teach, but rather treats the material as if it were actually part of the story of a typical third person shooter video game. Immune Attack is a video game. It does not break its cover; there are no quizzes to pass, no mini games, no guarantee of formal learning. Learning happens within the context of the game: Players only need to learn how to win the level in order to move on. Like a regular, commercial game, the player learns what they need to know and uses it immediately.

There is no explanation of what a protein is, instead we just get instructions to activate these proteins and we what results. We see a response by the cell to the activation of a proteins and we learn a concept. Therefore, Immune Attack presents a unique opportunity to begin to uncover what kind of learning and what kind of attitudes are imparted about biology through a commercial type video game.

Additionaly, it should be noted that Immune Attack is designed for gamers. The tutorial is not intuitive for anyone who has not played this kind of complex video games already. Created in 2007–2008, Immune Attack is similar to what is referred to as a “hardcore” video game. It expects the player to devote a few minutes to listening, learning and practicing the controls. In the 6 years since Immune Attack was released, advances in tutorial design and game design have resulted in new ways to involve the player in the story while giving them time to practice the controls. So the tutorial is more integrated in the game and therefore more engaging for the player. Immune Attack therefore, provides an excellent example of a game designed for a game playing audience. Any results we find for non gamers may very well be improved upon by a more intitive game design and more engaging tutorial.

Originally intended to teach immunology concepts to advanced high school students and beginning college students, Immune Attack teaches much more general content that immunology, rather it teaches the basics of molecular behavior. For example, Immune Attack shows players that individual proteins perform their own tasks and that proteins are not interchangeable. Additionally, Immune Attack shows that cells require a certain set of proteins in order to function and that missing a protein’s function can cause disease. Additionally, 6,000 grade school and high school teachers had registered to evaluate the game in their classrooms. Clearly, there was demand for the game fr use with younger students. Therefore, we decided to evaluate the effectiveness of Immune Attack as a teaching tool for basic cellular and molecular biology with students in 7th – 12th grades. (We present here our data from 10–12^th grade students. We will present our data on 7^th – 9^th grade students elsewhere.)

We address the following questions.

1. Can a video game (Immune Attack) teach concepts and terms of molecular biology, cellular biology and immunology?

2. Can a video game benefit students who do not play video games or who perform poorly in the test game?

3. Can a video game impart confidence, or in other words, a feeling of self-efficacy as regards the diagrams and graphics used to represent molecular and cellular biology data?

We tested Immune Attack players using their randomly assigned classmates as a control group. We developed a assessment of terms and concepts as well as an innovative test of confidence with molecular science. We found that Immune Attack players perform significantly better than control students on our test of terms and concepts. Additionally, players who reported playing 1 or more hour of video games per week, that is two-thirds of the students we tested, gained significant amount of confidence with diagrams related to molecular science. On the other hand, students who were not gamers, who performed poorly in the game and who claimed the game was hard to play showed no significant difference in confidence from the control students. However, while each of these groups (non-gamers, poor Immune Attack players, and those who reported that Immune Attack is hard to play) still had an average score that was significantly greater than control on the terms and concepts test. That is, all students responded well to our test of terms and concepts, while most gained confidence with the material. And most noticeably, this complex game did not cause any players to lose confidence in their molecular biology abilities. Therefore, a complex game like Immune Attack may well be a useful medium for molecular science education.

Experimental Methods

We built an assessment tool to measure 1) Knowledge gains in the area of cellular and molecular immunology, 2) Understanding of the game mechanism and 3) change in self confidence regarding the subject matter and 4) voluntary self reporting of demographic information. A complete list of these three sets of questions is available here: (Submitted as Electronic Supplementary Material :Survey Questions.xlsx) The assessment tool is taken –all three parts— by the students online. The online test takes about 30–40 minutes to complete. We created our own tests because it was completely unknown what we might discover. We tested the game in junior high and senior high schools across the US. We tested the game in classrooms not because the game is designed for use in a classroom, but because we needed to know the knowledge level of our test players and control players were as similar as possible.

Terms and Concepts

We developed a 27 item multiple choice test of terms and concepts to test for gains in knowledge of molecular and cellular biology. We started with basic questions and received such encouraging results we spent the two years creating harder questions. We tested hundreds of students in grades 7–12 in American schools. Our assessment questions were tested numerous times over the course of two years and in collaboration with statisticians Caroline Pinkham and David Silvernail at the University of Southern Maine, we had a multiple choice test with 27 items. The statistical report of details about the test of terms and concepts is included in the supplementary materials. The statistical report on the final iterations of this test are attached as supplemental materials. The final Cronbach alpha score for these 27 knowledge questions is 0.79.

Choosing the concepts to focus on

During the test creation phase, 2009–2010, we asked students to answer open ended questions, similarly to the method of Mike Klymkowsky (REF). The method of open ended question asking is designed to bring out student’s misconceptions. We used the students misconceptions to focus our assessment questions, since there are so many possible topics to focus on in the game. For example, when we asked students, “What is protein?” they often answered that “protein is found in chicken and egg whites? And that protein is an “important building block of the body.” In other words, is appeared that students knew about proteins generally, but did not realize that each protein is unique and has a particular job to do. The game Immune Attack requires players to locate and shoot particular proteins, Selectin and ICAM in the very first level of the game. So we used this to design our test questions. There are basic knowledge questions to determine whether players remember the names of two proteins required to complete level 1. Knowing that proteins have individual names and activities is the first step to understanding the bigger concept that mutations in a single gene can lead to disease. Questions in this group we refer to as One Protein/One Job. Other concepts are the process of Transmigration, the process of Cytokinesis and the meaning of Cell Differentiation. The survey questions are submitted as supplementary material.

Game Mechanism

An understanding of the game mechanism is expected to be a prerequisite to learning anything from the game, and lack of game mechanism understanding may explain a lack of learning or confidence gains. Therefore, we also created a 10-item multiple choice test that is designed to test whether players know what the goals of the first three levels. These questions simply ask the players, what is the name of this object and what is in this image? We included questions about objects required for game success and objects that were not required for success."What color are the monocytes?" Additionally, we showed images from the game such as the one in Figure 1C and asked "What is this arrow pointing to?"

Fig 1.

Immune Attack screen shots. A. Level one is about the process of transmigration. A. The Monocyte “the big blue ball” will roll over the Selectin proteins and slow down if the player shoots the Selectin proteins. Selectin is shown in panel C with the white arrow drawn in to indicate the yellow Selectin protein.

Confidence with Cell biology/biochemical diagrams

Our test of self confidence is a series of images, diagrams and photos that are typical of the images scientists use to communicate their research results as well as college level immunology textbooks. Rather than ask students the typical questions about whether they see themselves as scientists we wanted to ask, “Does this look like something you would read?” We reasoned that students do not know what the words “molecular biology”mean, so they can’t say whether they like it or would want to practice it or learn about it. Additionally, the game Immune Attack does not introduce a realistic science scenario and is not intended to present the career or activities of a scientist, rather all of the game takes place at the cellular and nanoscale levels. Immune Attack is, however, designed to present cell and proteins in a manner typical of scientific diagrams. Cells are drawn to mimic schematic drawings used in journal articles and textbooks. Therefore, we reasoned that players would become more familiar with such diagrams and perhaps gain a feeling of self efficacy when presented with such diagrams in the future. So we designed questions that shows the player cellular immunology related and see if playing a video game about the topic would affect their feelings of self confidence with the material. The five confidence questions ask students to agree or disagree with statement "I would be able to understand this if I read it and thought about it." See Figure 5 for the images used. See Results and Discussion for more detail.

Fig 5.

Confidence Questions. Five images were presented to Immune Attack and Control game players. Under each picture was the statement, “I would be able to understand this is I read and thought about it.” The five answer choices were 1) I agree completely, 2) I agree somewhat, 3) I am neutral, 4) I disagree somewhat and 5) I disagree completely.

Demographic information

As with every portion of this test, reporting of demographic information was voluntary. Students self reported their grades in science, math and English, hours per week spent playing video games and which kinds. Students who played Immune Attack were also asked whether the enjoyed the game, found it “easy to play”, would like to play such games in their science classes and whether they would recommend Immune Defense to their friend.

Protocol

Teachers reach us though our website, ImmuneAttack.org. We sent emails to 6000 people who registered as interested teachers prior to the game’s release in May 2008. We received responses from about 100 teachers. Upon informing those 100 teachers of our three day controlled protocol, ten became our first in-class evaluators. Over the next 3 years, 2009 through 2012, several teachers participated each year.

Controlling for student past experiences

Teachers were instructed to divide each of their classes randomly into two halves. Students were to play either Immune Attack or a control game for 40 minutes. The next week the same halves played the games again. Twenty-four hours after the second play period, all students took our online test of cellular immunology, game mechanisms and self confidence. Finally, teachers filled out another online survey in which they described in their own words how they divided their classes, how long the students played, and whether any deviations from the protocol or technical difficulties, etc. occurred. We were able to exclude data from teachers who did not describe a random method of dividing their class or who had technical difficulties playing the games. Teachers also distributed and collected consent forms. Our consent form explained that the games would be used in class and that all students can participate, but no data would be used from students who did not hand in their signed parental consent/student assent form.

The control games were related generally, but did not address any cellular or molecular biology: CSI The Experience (10–12th grades) or N-Squad (7–9th grades).By dividing each classroom in half we controlled for the background and educational experiences of the students as much as possible. Students self reported their grades in math, science and English, how many hours per week they play video games and which kinds.

To ensure that results were due to the game only, teachers were asked not to introduce the game or its topics. We also did not include any lecture or notes for teachers to read. We did however, provide teachers with a written step by step description of what players do in the first three levels. This walk through is useful for teachers who wish to follow up on what students learn in the game. The walk through, FAQ and curriculum guides are available at ImmuneAttack.org.

Results and Discussion

A statistical report characterizing the data collected is available as Supplementary Material. Additionally, a complete list of all the questions and their answers are available in Supplementary Material.

Immune Attack players were statistically significantly more likely to score higher on our test of terms and concepts than their classmates who played control games. Fig. 3A shows that the distribution of scores on our test of terms and concepts is shifted to the right. Fig. 3B.shows that boys and girls benefited from playing Immune Attack to the same degree. Girls scored on average 1 point higher than boys, but this difference is not statistically significant. So gender and prior video game experience did not have an effect on student scores on our test of cellular and molecular biology terms and concepts.

Fig 3.

Student scores on our test of Terms and Concepts presented in Immune Attack.

We were interested in knowing whether a third person shooter video game that requires two hands to control with a keyboard and mouse would perhaps intimidate or confuse non-gamers. We wanted to see if non gamers would perform as well as gamers on our test of terms and concepts. Fig. 3C.shows that students who report playing zero hours of video games per week scored as well as students who report playing 1–5 or 6+ hours of video games. Consistent with this figure the Pearson correlation and one way ANOVA calculations in the statistical report also indicate no statistically significant correlation among the high scoring group and the hours of games per week students reported playing, see supplementary materials.

There is no correlation with gender or with the number of hours students report playing video games each week. However, we still want to determine whether a video game would work for all students. So we looked at other criteria we though might affect students’ scores on the terms and concepts test. These criteria are how the players reacted to the game itself.

We assumed that students who did not beat the first level of Immune Attack would be a group least likely to perform well on our test. We designed the test of terms and concepts to draw from the first three levels of Immune Attack only.Figure 4A.shows the scores of Immune Attack players according to the level of the game they reported reaching. Our results show that all Immune Attack players, regardless of how many levels they reached, scored significantly better than the control game players.

Fig 4.

Scores on test of terms and concepts by A. level of Immune Attack reached and by B. response to the question “Immune Attack is easy to play.” Students were given images to select from in order to indicate which level of Immune Attack they reached—without needing to recall the name of the level. (See online survey, in supplementary martials.)

However, Immune Attack players who reached the second half of the game scored even better on the terms and concepts than the players who only reached the first half of the game. Because the test of terms and conceptst only covered the first three levels of Immune Attack, this difference is not due to the fact that level 5–7 players were exposed to more of the tested material. However, students who played through the second half of the game are exposed to more of the same concepts with new cells and new protein types (Fig. 4A).

Because learning from a video game is expected to require first an understanding of the mechanism of the game, we created a 10 question test. These questions about the game were straightforward: For example, students were show Figure 1C and asked “What is this arrow pointing to?” Control students were not asked these questions. Scores on the 27 terms and concepts test and the 10 game mechanisms test were very well correlated. There is a strong correlation between students’ score on the ten the game mechsnism questions and their scores on the test of concepts and terms (data not shown).

Additionally, responses to post-play questions such as do you agree that “Immune Attack is easy to play?” were correlated with higher scores on the Terms and Concepts question. So understanding the game mechanism and understanding how to play is important for remembering terms and concepts after the game.

Next we address whether the game imparts a feeling of confidence with the matrial presented. We reasoned that seeing the images of cells and proteins in the game would make players feel more familiar with these types of images. The cells are drawn similar to typical schematics scientists use to communicate their data, so we were able to use images used by scientists to convey their data with the game players in our study.

Figure 5 shows the five images used to test for student’s self confidence with images related to cellular and molecular biology. Students were asked to agree or disagree with statement "I would be able to understand this is I read it and thought about it."The following answer choices were available for each question:

1. I disagree definitely.

2. I disagree somewhat.

3. I am neutral.

4. I agree somewhat.

5. I agree definitely.

We found that Immune Attack players do gain confidence as regards the Transmigration and Data images. In the Transmigration image (5A), there is a high degree of complexity. This image, from NatureImmunology Reviews, is designed for an expert audience. The second image, “Yellow Macrophages” (5B) from Janeway’s Immunobiology textbook, is designed for undergraduate and graduate level immunology and has less detail. When we consider the simpler image, we see in Fig.6B and 7B that both Immune Attack players and their control classmates responded to the simpler image in the same way. The students’ answers form two similar curves. Wehen we consider the more complex Transmigration figure (Fig7B and 7A) however, we find a clear difference between the Immune Attack players and the control students. The control students respond more negatively to the more complex figure. About 25% of the control students responded “I disagree definitely,” indicating that they do not think they would be able to understand this image. However, Immune Attack players responded to the complex figure almost exactly as they did to the simpler textbook image.

Fig 6.

Confidence data for the (A) complex transmigration diagram, the (B)simple text book yellow macrophages diagram and the (C) data image.

Fig 7.

A–F. Confidence data for the (A and B) complex transmigration diagram, the (C and D) simple text book yellow macrophages diagram and the (E and D) data image broken down into girls data and boys data.

We wanted to know whether confidence with related images would carry over to other image types. Images of neurons and of a three dimensional cartoon of a protein complex did not elicit a more positive response from Immune Attack players as compared to the control. It may be that the positive effect on their responses is very specific to the type of image and the drawing style. Immune Attack cells look very similar to the diagram in Fig 1, in which H&E stained features of the white blood cells are drawn, namely the nucleus shape and color. However, it may also be that the complex transmigration image was the only image complex enough to elicit a negative response from the control students.

We wanted to see if Immune Attack players could extrapolate their new understanding and confidence to actual data images. Figure 5E is from William Muller’s groundbreaking work, which showed that transmigration—the objective of level 1 of Immune Attack—is dependent on Selectin—the protein players need to shoot to successfully transmigrate their monocyte. The images in the panels will naturally look foreign, since the scanning electron microscope images are 3D, grey and do not show any of the schematic details shown in the game, like nuclei. However, we wrote a figure legend in simple language that uses terms from the game and also clearly describes the experiment being presented. Immune Attack Players consistently respond more positively than control game players about their perceived ability to understand the data images (Fig. 7A). For the Data figure, the effect size is small (0.19), because the average score does not shift greatly, while for the Transmigration figure, the effect size is medium (0.47).

So a difference between Immune Attack players and controls is found in their reaction to the Transmigration diagram and the Data with figure legend. Immune Attack players respond more positively on the whole to both images. To investigate which groups of students gained confidence, we looked at boys vs. girls in Fig. 7A–F.and then at gamers vs. non gamers in Fig. 8A–E.

Fig 8.

A–E. Confidence data for each of the five images, with data broken down into All students, students who report playing 0 hours of video game per week, those who report playing 1–5 hours, and those who report playing 6 or more hours per week.

Considering first the effect of gender for the Transmigration image, both female and male Immune Attack players responded statistically significantly more positively to the Transmigration image compared to matched controls, although the difference has a smaller effect size for girls than boys. For the Data with figure legend, the increase in confidence is not significant unless all 180 Immune Attack students are compared to the control. This is primarily because the averages between the test and control means is so small. However, in Fig. 6C and Fig. 7E we can see that the curve of student responses is shifted to the more positive after exposure to Immune Attack. This shift effect is stronger for girls than for boys.

Considring the effect of hours of game play reported per week we find in Figure 8A that non gamers (0 hours per week reported) do not differ from non gamer control students. Students who report playing 1 or more hours per week, which is two thirds of the students tested, show a statitically significant shift in their responses to the Transmigration image. For the Data image, results ar only significant when all hours of games per week are considered together.

It is interesting to note, gamers (students who report playing 6 or more hours per week) in the control group were consistently more negative about their ability to understand these images. Perhaps video gamer players can more accurately assess their ability to understand a new diagram Fig. 8A–E.

Finally, we looked at whether students’ scores on the terms and concepts test are correlated with their confidence scores. We found that the students who scored 9–16 and 17–27 questions correct also responded more favorably to the Transmigration diagram and the Data images. The confidence measurement for students who scores 8 or fewer correct answers on the terms and concepts test could not be distinguished form the control response.

Conclusions

We have found that students learn molecular and cellular biology concepts and terms by playing a video game set in the molecular world. Even though the game does not explicitly teach these terms and concepts, students can answer questions on our 27 item multiple choice test significantly better than their classmates who played an unrelated video game. The higher average scores on the test of terms and concepts was true for all Immune Attack players regardless of how many hours a week they play games and whether they are male or female. We found no factor that can predict prior to playing the game whether students will perform well on the test of terms and concepts. However, we found that students who scored in the bottom 24% on the test of terms and concepts were more likely to report that the Immune Attack is not easy to play. Additionally, understanding the game mechanism questions was the strongest indicator of whether students scored well on the test of terms and concepts. We conclude that a video game like Immune Attack--a shooter styled, self directed game with minimal explicit learning activities--can improve student scores on a test of terms and concepts. Importantly, a game with a more engaging introduction to the controls and game mechsnism than Immune Attack may be even more effective at helping students remember terms and concepts.

Immune Attack also had a positive effect on the confidence players feel in their abilities to understand complex diagrams related to the game. However, unlike the test of terms and concepts, we found a difference between gamers and non gamers. In particular, we found that students who reported playing zero hours of video games per week showed no difference in confidence with a molecular biology diagram as compared to the control students. So there were no confidence gains in the non gamer group. However, Immune Attack is a complex third person shooter, in which players need to learn to navigate a microbot in three dimensional space, avoid many dangerous objects and shoot a ray gun at moving proteins. Yet, despite the complexity of the game, non gamers do not show any loss of confidence compared to the control group regarding molecular biology. We did not find a difference in confidence between the gender groups, however. Boys and well as girls showed a statistically significant shift in their response to a complex scientific diagram.

Additionally, the majority of students who reported enjoying Immune Attack and who played past the third level of the game showed the greatest increase in confidence, regardless of whether they were gamers. Most interestingly, when presented with two molecular cell biology diagrams, one designed for experts and one designed for students, Immune Attack players responded to the expert diagram as favorably as if it were the simpler one.

We have shown that a large fraction of Immune Attack players gained confidence with the kinds of diagrams scientists use to communicate their data, while not causing a loss of confidence in non gamers or students who did not perform well in the game. Immune Attack was successful at increasing the average score of all students, gamers and non gamers alike, on our test of terms and concepts. Therefore, we conclude that a complex video game like Immune Attack that 1) requires a tutorial to teach the game mechanism similar to a hardcore video game and 2) teaches through story based interactions with a wide array of objects but without explict text based questions to answer cn increase the scores of players on a test oc terms and concepts, increase the confidece of gamers with the material and not harm the confidence levels of non gamer students. We sugest that game design improvements can positively affect the confidence levels of the non gamer students.

Fig 2.

Schedule for classroom evaluations. Our evaluation protocol consisted of two 40-minute play periods and an online post test. Teachers reported to us through our online survey, filling in their own words how they accomplished the random assignment of their students to the two groups, how long the students actually spent with the games each day and if there were any technical problems.

Fig 9.

Confidence data for each of the five images, with data broken down by how well students performed on the test of Terms and Concepts.