Adolescents’ knowledge of and confidence in vaccines improves with board game play

Abstract

A long history of misinformation has led to anti-vaccine sentiment, risking everyone’s health. Despite public health campaigns, vaccine hesitancy rates have not declined. In response, we created an educational board game to address the fundamental science of vaccine development. The cooperative game guides players through identifying a novel pathogen, developing a vaccine, conducting animal testing, running clinical trials, and distributing doses to the public before the virulence takes over. Players need to work cooperatively, sharing resources in order to win the game; everyone wins or loses together. Developed in consultation with vaccine development researchers, the game retains many of the real world scientific and clinical challenges that face vaccine developers. Assessing students in six different communities in four different mid-west and eastern states in the United States provided access to a diverse demographic. We find that through a pre- post-assessment, students (grades 7–11, n = 304) show significant gains in vaccine knowledge and confidence, demonstrating that fun, gamified learning can improve health literacy. After gameplay, students recognize that the soft skills of being a good communicator, being creative, and sharing resources are important for being a scientist.

Supplementary Information

The online version contains supplementary material available at 10.1007/s44217-025-00683-4.

Introduction

Vaccine hesitancy has a direct effect on global human health [1]. While it may seem that the phenomena is new, it is not. To address vaccine hesitancy, we explore how an educational board game can improve understanding of the efficacy and safety of vaccines. Additionally, we investigate how the game can impact players’ appreciation and understanding of the “soft skills” of collaboration, communication, creativity, and resource sharing that are necessary for scientists to succeed. This investigation used a purpose-designed game entitled, N.O.V.E.L. (Newly Observed Variant of Extreme Lethality), created by the authors [2] (Fig. 1).

Fig. 1.

The board game N.O.V.E.L. (Newly Observed Variant of Extreme Lethality) [2]. A Contained in the game box is a variety of components, including cards, dice, game boards and other pieces. For use at home, or in the classroom, the game is designed for ages 10 and up and can support up to 4 players. B Phase 3, animal testing, is displayed. Supporting educational curriculum for the game can be found at www.ThePartnershipinEducation.com

A brief history of disease, vaccines, and mistrust associated with COVID-19

We know that awareness of infectious disease has been evident throughout human history, with various notable descriptions by ancient Egyptians [3]; the Greeks, Hippocrates and Thucydides [4, 5]; Ibn Sīnā of the Samanid Empire; and the Moroccan scholar Ibn al-Haj al-Abdari, among others [4, 6]. The observations of microorganisms by Anton van Leeuwenhoek and Robert Hooke in the 1600’s, plus studies done by Louis Pasteur, Robert Koch and others working in the 1800’s, led to the definition of the germ theory of disease [7].

Intentionally creating immunization to diseases has a long history dating back to at least the tenth century in Asia, with the inoculation of healthy people with smallpox by using material from pustules or scabs derived from smallpox patients with mild symptoms [8, 9]. The technique slowly caught on in the West as knowledge spread with the expansion of the spice trade, the Enlightenment, and Imperialism [10–14]. The English physician and scientist Edward Jenner, proved [8] that inoculation with cow pox was effective [6] at preventing small pox, yet considerable public and professional hesitancy persisted. This is due, in part to the treatment being considered foreign, resistance from the hierarchical medical and scientific communities, as well as certain theological considerations that variolation was against divine providence [13, 15]. Even in the face of mixed public opinion of inoculation treatments, governments began to see a public health benefit.

As early as 1806, governments began to appreciate the importance of mandatory inoculation for university students [16, 17]. Following inoculation successes in Boston, the Massachusetts legislature passed the first immunization law in the U.S. for smallpox vaccination in 1809[16], followed later by the United Kingdom [16]. For over one hundred years, there were varying degrees of acceptance on these and other regulations combined with a persistent undercurrent of vaccine hesitancy due to science skepticism as well as consideration of liberties and privacy [18, 19]. Despite this, vaccinology flourished with the development of over 30 vaccines [18] including those for rabies and anthrax in the 1880’s [20, 21], diphtheria and tetanus in the 1920’s [18], and yellow fever in the 1930’s [22]. A vaccine for whooping cough was combined with vaccines for diphtheria and tetanus to make a combination vaccine called DPT in the 1940’s[23]. In the 1950’s a polio vaccine was developed by Jonas Salk (1955), and the 1960’s saw vaccines for measles, mumps, and rubella that were combined to make the MMR vaccine, among others [18]. As a result of vaccination efforts, the global eradication of smallpox was announced by the World Health Organization in 1980 [24] and several other diseases are nearly unheard of in developed countries such as polio and measles [25]. In just the last 50 years, it is estimated that over 150 million lives have been saved because of vaccinations [1, 25, 26]. Not only are lives saved, but significant economic benefits are realized. Studies estimate that the economic benefit to the United States is on the order of $70 billion per cohort of children born in a year (for the study those born in 2009) [27] and further, that in low and middle income countries immunization programs saved nearly $600 billion by reducing costs of illness and $1.5 trillion when broader economic benefits are considered over the period of 2011–2020 [26, 28]

With these profound successes and growth in public acceptance, a significant turning point in hesitancy came with the criminality of Andrew Wakefield’s publication of fraudulent data associating the MMR vaccine with bowel disease and autism. The determination that he was guilty of intentional fraud and driven by profit and fame did not undo the damage [29]. Numerous investigations and studies proved that Wakefield fabricated his data and findings, but nonetheless this negative association was established, and its impacts persist today[30, 31].

The COVID-19 pandemic and the impact of vaccine hesitancy

The coronavirus, SARS CoV-2, emerged in 2019 and spread globally, resulting in the COVID-19 pandemic. With the virus’s genomic sequence released on January 11, 2020, several multi-national research institutions, pharmaceutical companies, and governmental organizations collaborated to expedite the creation, testing, validation, and production of vaccines [32]. The new vaccines became widely available within a year of the pathogen’s discovery. In early 2022, the impact of the vaccine became evident. While the Omicron variant was dominant, hospitalization rates were nearly 11 times higher for unvaccinated persons [33] and the likelihood of death was 10 times higher [34] when compared to those who were vaccinated. Still, vaccine hesitancy persisted as a result of unfounded concerns, misinformation, conspiracy theories, demographic variables, and socioeconomic conditions [35].

The availability of vaccine safety and efficacy information for the general public did not increase the likelihood that a vaccine hesitant individual would get vaccinated [36]. Notably, two factors; ease of vaccination access and conversations with trusted family and friends, showed positive impacts on vaccination rates [36]. Chris Mooney and Sheril Kirshenbaum wrote in their book, Unscientific America, How Scientific Illiteracy Threatens our Future, that “people integrate new information based on their pre-existing worldviews, and that failure to account for this fact will lead to continued failures in science communication.” [37] As such, one way forward in addressing vaccine hesitancy is to first appreciate the world view of the audience.

A way forward

Our efforts centered on addressing stressors associated with adolescents aged 10–18 and found that this group harbors concerns about their own health and their family’s health. We revealed this by conducting a small-scale survey in 2019 with the aim to understand what facets of the adolescents’ lives were sources of stress and anxiety. Initially completed prior to the emergence of covid, we decided to repeat the survey during the pandemic lockdowns in 2020, thus gaining a fortuitous pre- post- snapshot of adolescents’ attitudes. We also asked the adolescents’ parents and teachers what they perceived to be the stressors of their children/students. We found that during the pandemic, adolescents' concern about their own health was 66% higher than what parents thought. Similarly, students scored their stress about the health of others 62% higher than the adults' perception (Tables S1-S5). From this, we conclude that students have significant concerns about key health issues relevant to themselves and their families. We then set out to address these concerns in a manner that could help foster a stronger understanding that vaccines are safe and effective. We determined that a tabletop board game about the process of vaccine development could provide an appropriate learning opportunity about vaccines, so we created the cooperative board game N.O.V.E.L [2]. (Fig. 1).

N.O.V.E.L. provides an understanding of vaccines and their safety by demonstrating the robust, science-based safeguarded development process they undergo. The game simulates the multi-phase process that scientists use to create vaccines. In-game, players must save humanity from a mysterious, fictional disease that is spreading across the globe. Together, players work to research the pathogen and identify its form (virus, bacteria, parasite, mold or fungus), create a vaccine prototype, test their prototypes first with animal models followed by clinical trials with humans, and finally distribute their successful vaccine to the public, all before the deadly disease takes over. Along the way, players manage resources like money, and time, which they can strategically share with other players to reach their mutual goal and win the game.

The game was developed by Duquesne University’s Partnership in Education, with collaboration from vaccine experts from the Center for Vaccine Research at the University of Pittsburgh School of Medicine and Director Paul Duprex, Ph.D. As the game’s developers, investigated N.O.V.E.L.’s impact on players’ understanding of vaccine safety and efficacy. We also explore N.O.V.E.L.’s impact on players’ understanding of science as a collaborative endeavor that requires pro-social “soft” skills such as communication and creativity [38–40] as well as collaboration and resource sharing.

Research questions

Collaborating with independent evaluators at Rockman et al. Cooperative, we investigated N.O.V.E.L.’s appeal, feasibility, and value in 7th-11th grade classroom settings. The overall purpose of this study was to evaluate how N.O.V.E.L [2]. could help adolescents learn science concepts. Our research questions include:

1. Does playing N.O.V.E.L. impact adolescents’ knowledge of vaccines and their development process?

2. Does playing N.O.V.E.L. increase adolescents’ awareness of the soft skills required by scientists?

3. Does playing N.O.V.E.L. increase adolescents’ perception of the safety and efficacy of vaccines?

To answer these questions, we conducted a mixed-methods study with a single arm before/after gameplay survey and report the aggregated quantitative and qualitative data from seven teachers and their adolescent students (n = 304). We also provide a case study narrative that explores one rural teacher’s experience in greater depth. Taken together, the findings document N.O.V.E.L.’s short-term impact of improving knowledge of and confidence in vaccine efficacy and safety. This study illustrates the value of educational games as effective methods for teaching contemporary health issues.

Methods

The game—N.O.V.E.L. newly observed variant of extreme lethality

Developed by the Partnership in Education at Duquesne University, N.O.V.E.L.-Newly Observed Variant of Extreme Lethality (N.O.V.E.L.), is an educational, table-top board game that takes players on a journey through the science of the vaccine development process (Fig. 1)[2]. Designed for 2–4 players, ages 10 and up, N.O.V.E.L. is a cooperative game where players work together as a team of scientists who combat the spread of a new infectious disease. Played in five phases, players identify the type of pathogen (virus, bacteria, mold, fungus or parasite), create a prototype vaccine, test in animals and then humans to finally distribute it. To win, players must successfully develop and distribute a vaccine before the disease infects all of humanity. Failure to launch a vaccine will result in the end of humanity and a loss of the game.

Study design

We conducted a mixed-methods study with a single arm before/after gameplay survey design (complete surveys can be found in the Supplementary Materials). This approach is appropriate for evaluations seeking to demonstrate the initial promise of an intervention [41]. We report the aggregated quantitative and qualitative data from seven teachers and their students (n = 304) and a case study narrative that explores one rural teacher’s experience in greater depth. The communication with teachers and primary data collection and analysis was facilitated by the independent evaluators at Rockman et al. Cooperative.

Teacher recruitment

Institutional Review Board (IRB) approval included that for all research involving human subjects, freely-given, informed consent to participate in the study and consent to publish would be obtained from participants and that for children under the age of 18, their parent or legal guardian provided written consent and that the child provided written assent to participate and consent to publish. The review of the IRB protocol was conducted in accordance with the ethical principles of the Belmont Report by Heartland IRB who provided approval (file number HIRB Project No. 10052023-515; Clinical Trial Number—not applicable) 7th-11th grade teachers were recruited between October 2023 and May 2024 via email and a digital flyer. Interested teachers completed a screener survey to determine eligibility for participation in the study. Teachers who met the study’s inclusion criteria attended an information session about the expectations of and incentives for teacher participation. Once a teacher received approval from their school principal and signed a consent form, they were enrolled in the study. Teachers then facilitated the recruitment of the students by communicating with parents, getting parental consent and then getting student assent forms completed.

Teachers and schools

Seven teachers from six middle and high schools took part in this study, spanning four eastern and midwestern states (Fig. 2). One participating school was private, and the rest were public. Of the 6 schools, 3 were middle school for grades 6 to 8, and three were high schools for grades 9 to 12. However, most of the students who participated in this study were 7th and 8th grade students. The schools ranged in size from 250 to 1200 students, most of whom were white (Table 1). One third or fewer were eligible for free or reduced lunch, except for one school where 86% qualified. Between 12 and 32% of each school’s population received special education services.

Fig. 2.

Geographic distribution of participating schools

Table 1.

Summary of participating teachers and the demographics of their schools

School (pseudonym)	Teacher(s) (pseudonym)	Number of students in study	School level characteristics
			Number of students at school	Racial/ethnic distribution	% Free/ reduced lunch	% IEP*/ Special ed
Apple Valley Middle School	Tiffany	37	1000	White: 71%, Black: 8%, Latino/a:11% Asian: 2%	31%	14%
Big Dipper High School	Matt, Kathy	47	1200	W: 75%, B: 5%, L: 7%, A: 6%	20%	12%
David High School	Valerie	28	300	W: 59%, B: 16%, L: 18%, Other: 7%	36%	15%
Knoxville High School	Patsy	44	550	W: 97%, B: 2%, L: 1%	44%	20%
Lake Middle School	Stacie	104	250	W: 99.6%	86%	32%
Winding Road Middle School	Kelsey	44	400	W: 61%, B: 11%, L:3%, A:11% Mixed: 9%, Other: 5%	X	X
		304

*Individualized Education Plan (IEP)

Student participant characteristics

The study included 304 consented students. Of those, 263 (86.5%) submitted matching before and after gameplay surveys. About two-thirds were middle school students in grades 7 and 8 (Table 2). The sample was evenly distributed between females and males. Twenty-one percent belonged to races or ethnicities underrepresented in Science Technology Engineering and Math (STEM) disciplines, as defined by the National Institutes of Health (2019): Black/African American, Hispanic American/Latino(a), Native Alaskan, Native American, and Pacific Islander, or any combination.

Table 2.

Demographics of participating students (n = 304)

We asked students how often they played board games or card games since that might affect their experience with N.O.V.E.L. About half played at least once a month, while one-third played once or twice a year (Fig. S1). Only 6% never played board or card games.

Data collection

We asked teachers to complete a list of tasks including the following: (a) obtaining principal, parental, and student consent, (b) implementing gameplay, (c) administering before/after gameplay student surveys, and (d) talking to the independent evaluators from Rockman et al. Cooperative about their experience. Project staff provided teachers with one copy of the game for approximately every four students in their largest class period. Teachers could ask students to follow the written directions contained in each game box or show segments from a 25-min video. Students’ gameplay lasted three to five days depending on the teacher. All teachers played the game in their science classes except for one teacher who implemented the game in her high school economics class. To thank teachers for their efforts, we gave them a $300 stipend and let them keep the games. The students were not given any incentives and played the game during class-time.

Student surveys

Students answered questions about study outcomes, their gameplay experience, and their personal background summarized in Table 3. Further information about item scoring is included (Table S17) with full copies of the survey instruments at the end of Supplementary Materials.

Table 3.

Summary of before and after gameplay survey contents

Topic	Sample item(s)	Number of items	Item format	Item scoring or response scale
Vaccine production knowledge	Which of the following are types of vaccines? Select all that apply (list of 7 options such as mRNA and antivirals) When would scientists begin testing their vaccine prototype in animals? • When the vaccine prototype becomes cheap to produce • As soon as they can put together a combination of ingredients (correct answer) • After they find the right ingredients that produce an immune response • Before the disease has had a chance to spread	10	9 multiple choice, 1 open-ended	41 points total: 9 items worth 4 points each; 1 item worth 5 points
Soft skill importance	How important are these skills in being a good scientist? • Being a good communicator • Being independent	8	Fixed choice, Likert scale	1 = Not at all important; 6 = Extremely important
Perceptions of vaccine safety and effectiveness	Rate how much you agree or disagree with the following statements • Human testing of vaccines helps to ensure that new vaccines are safe to use • I am concerned about the safety of newly developed vaccines	6	Fixed choice, Likert scale	1 = Strongly disagree; 6 = Strongly agree
Reactions to N.O.V.E.L. (post only)	How would you rate your enjoyment of the game, N.O.V.E.L.? How important are these skills while playing N.O.V.E.L.? • Being a good communicator • Being independent In your own words, what is one thing you learned from playing N.O.V.E.L.?	11	9 fixed choice, 2 open-ended	Varies by item
Background (pre only)	What grade are you in? How often do you play board games or card games?	5	3 fixed choice, 2 open-ended	Varies by item

Survey items were sourced from existing, validated instruments where available and were supplemented by self-drafted items when necessary. Self-drafted items were reviewed by evaluation experts, and aligned with recommended guidelines [42, 43]. These items were also validated by middle school students, asking them to think through their answers and identify any words or phrases they did not understand [44]. The students validating the survey were a different cohort than the ones used in data collection. Student feedback was used to eliminate items that they could not answer easily even before playing the game.

We created the soft skills items by identifying the abilities that students would need to use in the game (e.g., sharing resources, communication), and the misconceptions we hoped to dispel (e.g., that scientists work alone and compete with one another). We selected a six-point rating scale—a survey design best practice—to be long enough to detect potential pre-post changes, but short enough for students to answer easily. We also labeled each point to help students apply these ratings consistently [45].

Finally, after reviewing published instruments on adults’ attitudes toward vaccination [46–48], we created our own items about students’ perceptions of vaccine safety and effectiveness. We measured some of the same ideas about safety and effectiveness but chose to write simpler items targeted for a secondary school audience. In some cases, we turned some of N.O.V.E.L.’s key takeaways into survey items, such as “Lots of research goes into developing a vaccine before it is released to the public.” We eventually settled on six items for students to rate on a six-point Likert response scale (1 = Strongly disagree, 6 = Strongly agree).

Teacher interviews

We interviewed each teacher on Zoom for about 30 min at the end of the study. During this conversation, teachers described the class setting in which they tested N.O.V.E.L. and shared students’ reactions to the game. Teachers also offered their own feedback about the game and whether they might consider using it again. We recorded the interviews and transcribed them for analysis.

Analysis

Quantitative

We analyzed the quantitative data with descriptive statistics, such as means and frequencies, and inferential statistics, such as paired and independent-sample t-tests and one way Analysis of Variance (ANOVA). When we ran batches of tests, such as when we examined all the vaccine items at once or looked at differences in outcomes by demographics (grade, gender, race/ethnicity), we lowered our threshold for statistical significance. This helped us minimize the likelihood of “false positive” significant results [49]. We applied a Bonferroni adjustment by dividing the standard alpha of 0.05 by the number of statistical tests we performed. For instance, with the six vaccine attitude items, we considered any p values below 0.008 to be statistically significant (0.05/6).

We also calculated a standardized effect size for each test to complement the p values [50]. These statistics enable us to describe the practical significance of each result, and to compare outcomes measured on different scales[51]. We report the Cohen’s d effect size for all t-tests. Traditionally, Cohen’s d values of 0.2, 0.5, and 0.8 represent small, medium and large effects respectively. These are only meant as rules of thumb; interpretations depend greatly on study design and context. In evaluations of educational interventions, effects of 0.2 are quite common and may still be considered meaningful [52].

Qualitative

We approached most of the interview and open-ended survey responses from the “bottom up,” identifying codes and themes from the data we collected[53]. We then counted the frequency of the survey codes. For a couple of open-ended items, including one testing vaccine production knowledge, we applied a rubric that the Partnership in Education game developers had created.

Results

The game N.O.V.E.L. has high playability and educational impact. Outcomes include that students (n = 263) significantly improved their knowledge about how vaccines are made (t(262) = − 10.15, p < 0.001, Cohen’s d = 0.63). On average, their scores increased 13.6% (3.36 points) from before to after gameplay (Fig. 3A, Supplement Table S6). Follow-up analysis suggests that students recognized many of the vaccine design elements featured in N.O.V.E.L., correctly identifying more vaccine types, vaccine ingredients, and animal model species used to test vaccines after playing the game (Fig. 3B, Table S7). We explored whether knowledge gains differed by students’ grade, gender, race/ethnicity (Table S8 and Table S9), or general frequency of at-home board game play (Fig. S1). We also considered whether students who won the game performed better than those who lost or did not finish. No significant differences in performance were found based on gender, race/ethnicity, win/lose status or frequency of at-home board game play (Supplement Tables S8 and Table S9). It was, however, evident that the middle schoolers started at a lower overall knowledge point than did the high schoolers, so they had greater opportunity to gain knowledge (Fig. 3C). Importantly, after gameplay students indicated greater confident that their answers were correct (Fig. 3D).

Fig. 3.

Student gains in knowledge and confidence after playing N.O.V.E.L. A. Students’ mean vaccine knowledge scores before and after playing N.O.V.E.L. (n = 263 students, Table S9 for t-statistics and effect sizes). B. Students’ identification of vaccine types, ingredients, and testable species before and after playing N.O.V.E.L. (n = 263 students for types and ingredients, and n = 262 for species, Table S7). C. Mean change in students’ vaccine development knowledge scores by grade level (n = 263 Table S8 and Table S9). D. Students’ confidence in their knowledge of vaccine development before and after playing N.O.V.E.L. (n = 263 Table S10, Table S11). Response scale: 1 = Not at all confident, 2 = Slightly confident, 3 = Somewhat confident, 4 = Moderately confident, 5 = Very confident, 6 = Extremely confident. E. Students’ mean agreement with statements about vaccine safety and efficacy. Scale: 1 = Strongly disagree, 2 = Somewhat disagree, 3 = Slightly disagree, 4 = Slightly agree, 5 = Somewhat agree, 6 = Strongly agree. Statistically significant before/after gameplay difference p < 0.001. • Statistically significant pre- post difference. (Supplement Table S12)

Perceptions of Vaccines Even before playing N.O.V.E.L., students generally agreed that vaccines were safe and effective (Fig. 3E). They also knew that vaccines didn’t generally contain harmful substances. And importantly, they knew before playing the game that healthy people need vaccinations; universally disagreeing with the statement that ‘healthy people do not need to get vaccinated.’ After playing the game, they felt more positive about the value of human testing. This change was statistically significant (t(262) = 3.23, p < 0.001, Cohen’s d = 0.20). There were no significant differences found by grade, gender, or race/ethnicity. Open-ended responses reveal that students also developed a deeper appreciation for how vaccines are made. They learned that vaccine development is a multistep process that is more difficult and complex than they thought (Table 4).

Table 4.

Concepts and skills students reported learning from N.O.V.E.L. (n = 280)

Theme	% of Responses	Sample quotes
Making vaccines is hard. It requires a great deal of time and effort	25.0	“One thing I learned about playing N.O.V.E.L. was how much work or phases it takes just to make sure the vaccine will work on humans.” “I learned that it is hard to make vaccines while the pathogen is still spreading and being rushed to finish.” “I learned that the process from having a virus to creating a vaccine to sending it to the public have way more stops and it is more complicated.”
There are specific steps involved in producing vaccines (e.g., distribution, animal testing, prototypes)	18.9	“I learned about the process of the shipment and distribution of the vaccine.” “What animals they use for testing” “They do human test after animal”
There is a general, standardized process for making vaccines	14.3	“I learned the process used to make a new vaccine” “The stages of vaccine development.”
Playing the game requires teamwork and communication. You also need to follow rules	11.1	“The game is fun and it is hard if people don't talk” “To be cooperative and being able to follow the rules with vaccines and stuff”
Total	69.3

Note: 5% of students did not answer this question. Individual codes for the remaining 25.7% of responses accounted for no more than 4% each of the total responses

Scientists’ soft skills

We expected that playing N.O.V.E.L. would require players to practice the same soft skills that are needed by scientists in the real-life vaccine development process. To investigate this, we asked students to rate the importance of eight skills embodied in being a “good scientist,” before and after playing N.O.V.E.L. These included abilities such as being communicative or cooperative (Fig. 4A). We found that prior to gameplay, students already believed that most of the skills were very or extremely important. For example, ‘Being a good communicator,’ ‘Being cooperative’, ‘Being able to follow rules’, ‘Being creative’, ‘Sharing resources’ and ‘Collaborating on vaccines’ collectively had an average before-gameplay rating of 5.1 out of 6 and an average rating of 5.19 after gameplay. Following gameplay, students’ ratings of three soft skills increased significantly, including the importance of sharing resources and being a good communicator. The concept of being competitive was viewed as being slightly more important after gameplay increasing from 2.53 to 2.91 (Supplement Table S13 for t-statistics and effect sizes). None of these results differed by grade level, gender, race or ethnicity.

Fig. 4.

Changes in students’ perceptions of importance of soft skills before after playing N.O.V.E.L. A Students’ mean ratings of the importance of soft skills for being a good scientist before and after playing N.O.V.E.L. B Students’ mean ratings of the importance of selected soft skills for being a “good scientist” and for playing N.O.V.E.L., after gameplay. C. Middle and high school students’ ratings of the importance of creativity for being a “good scientist” and playing N.O.V.E.L., after gameplay. Scale: 1 = Not at all important, 2 = A little important, 3 = Somewhat important, 4 = Moderately important, 5 = Very important, 6 = Extremely important. See supplement Table S13 and Table S14 for t-statistics and effect sizes. • Statistically significant pre- post difference

To further explore students’ perception of the skills needed by scientists, we asked them to rate the importance of six of the skills used in playing N.O.V.E.L. as compared to those same skills for being an effective scientist. Students thought the skills of communication, collaboration, competition, and rule-following were equally important for both contexts (Fig. 4B, Supplement Table S14 for t statistics). Their ratings differed for being independent and for being creative, which were perceived to be significantly more important for being a good scientist than for playing the game (Fig. 4B, Table S14).

We also found a significant difference in ratings by grade level. High school students reported a bigger difference between the importance of creativity for playing N.O.V.E.L. and for being a scientist than did middle schoolers (Fig. 4C). The size of that difference (0.29 points for middle school students, 0.91 points for high schoolers) was statistically significant (t(256) = 2.94, p = 0.002, Cohen’s d = 0.44). There were no significant differences in ratings by gender or racial or ethnic background.

Appeal and value

Most students and all teachers thought N.O.V.E.L. was engaging and educational. Notably, 94% of students found that N.O.V.E.L. was fun (n = 278) (Supplemental Fig. S2). Students responded to the open-ended question “Is there anything else that you would like to tell us about your experience playing N.O.V.E.L.?” with comments that included:

• “I had a fun time learning about diseases and vaccines.”

• “It was a good bonding experience with my group of friends.”

• “Playing N.O.V.E.L. was a great experience. I would love to play it again if I could in the future."

Teachers were also very positive and satisfied with N.O.V.E.L. During her interview after gameplay, one classroom educator was asked if she would play the game with her students again. She stated:

“Yes, I would. My class perfectly lends itself to it and does it nicely…I appreciate the opportunity, and it was exciting.”

N.O.V.E.L. seemed to reach students who were normally disengaged with classroom activities. Different teachers noted:

• “I've never had some of these kids participate in anything, even labs. But in this game, they were participating and ensuring their group didn't get to 100 [lose], which was great to see."

• "They were definitely engaged. Everybody was participating the whole time. The kids who I didn't think would really be into it were probably the ones who enjoyed it the most. We approached it as learning the science process, and once they saw it wasn't hard, they had fun and gained confidence."

• "They got creative and engaged, especially when naming the diseases. Initially, some were frustrated, but by the second day, they were more connected.… One typically disengaged student wanted to participate actively in the game, which was a significant moment for her and a breakthrough in her social engagement."

Additional teacher reflections

Teacher interviews also offered insight into the dynamic and adaptive nature of game-based learning. Teachers felt N.O.V.E.L. offered meaningful learning opportunities about infectious disease and the vaccine development process. They appreciated the way the game diverged from more conventional teaching methods to better resonate with contemporary classroom environments and challenges.

Critical thinking around real-world situations

Teachers said they enjoyed this opportunity for their students because the game challenged vaccine misinformation and addressed relatable science issues.

• "The game provided a fun way to teach kids about vaccinations, which was particularly relevant given the misinformation around COVID-19 vaccines. It helped clarify these issues and educated them on the importance of vaccines.”

• "The game modeled the actual science behind creating vaccines, explaining each process step-by-step. This clarity helped dispel misconceptions about how vaccines are developed and tested."

• "We addressed real-world public health issues through the game, making the lessons more meaningful and relatable to the students."

• "The game facilitated discussions about real-life issues, helping students connect with each other and the content on a deeper level."

Differentiation and inclusivity

Teachers who implemented this game across multiple grade levels and disciplines noted the game's design strengths, such as physical durability, clear instructions, and visually engaging components. These game assets proved effective for both general and special education students when used with appropriate supports. Teachers shared examples of how students from a broad range of backgrounds and experiences could make meaningful contributions.

• "Even students who did not speak or read English well were involved, using translators and collaborating with others. This inclusivity was a significant positive aspect.… A lot of them wanted to take it home and play with their parents,.”

• "We used Google Translate for some instructions because we have students still learning English. This tool helped include everyone and ensure they could participate fully."

• "I don't have an inclusion teacher or aides, but we work with the special education department to support students during lunch breaks. This game helped integrate all students, including those needing extra help."

• "The game's collaborative aspect helped students learn to work together, appreciate different perspectives, and support each other."

• "The game's different roles allowed students to engage at various levels, making it accessible for all, including those who might struggle with certain concepts."

As noted for several of the assessed questions, gender, race, or ethnicity did not appear to influence this learning opportunity (Table S8 and Table S9). Even students who lost the game still learned nearly equal to those who won (Table S8). Similarly, it did not matter whether the students regularly played board games or were new to this style of gaming (Table S10). Furthermore, across all tested parameters, playing the game did not degrade any aspect of the student’s prior knowledge.

Discussion

Playing N.O.V.E.L. did increase students’ awareness of the resources needed in making vaccines and allowed them to learn aspects of the underlying science involved. Students also gained understanding of the attention to detail and safety that supports the development of new vaccines. Teachers reported that the game’s real-world premise engaged students from varied grade levels, abilities, and racial or ethnic backgrounds equally. Modest knowledge gains are to be expected for a short intervention of playing the game once.

Collectively, students’ knowledge scores improved 14% (Fig. 3A), which if brought into an academic setting would be almost a letter grade and a half. In the included case study of teacher Stacie’s cohort of 91 students in an economically disadvantaged rural school, we found that they improved their scores by 22% (Fig CS1B); an increase equivalent to 2 letter grades. Considering the example of the students understanding of animal testing, their score improved from 37% correct to an incredible 82% (Fig. 3B). We also found that middle school students experienced greater gains in knowledge about vaccine development than their high school counterparts reflecting that middle schoolers had a lower starting base knowledge than the high schoolers (Fig. 3C; Table S9). Overall, students also developed greater confidence in their knowledge. They also learned about the importance of communication and resource-sharing in science (Fig. 4). This is important because in this activity, instead of looking at a screen, a whiteboard, or pages in a book, or instead of listening to a teacher, the students are directly speaking with one another in face-to-face social banter, which has the capacity to provide a new learning dimension and reinforce social engagement. This represents a critical opportunity especially considering the residual impact of the distance-learning that was imposed during the pandemic. The social isolation and forced effort to learn through a screen interfered with students’ learning and their development of normal social engagement skills [54–57]. Here, the act of playing the game with face-to-face active cooperative engagement with peers shifts the balance of pro-social activities in positive ways.

Interestingly, while students’ knowledge about vaccines increased, attitudes toward vaccination did not change that much because our data show that they already thought vaccines were generally safe and effective. Even though there are individuals and pockets of certain communities where vaccinations are shunned, 88% of Americans, for example, generally accept that the benefits of MMR outweigh the perceived risks of vaccination [58]. In related studies, the Center for Disease Control and Prevention finds that MMR childhood vaccination coverage, while having dipped some due to the effects of the pandemic, are still consistently above 90% nationwide [59, 60], though the large outbreaks of 50 or more cases of measles is becoming more common [61]. However, even with this degree of acceptance for MMR, flu vaccination rates are only around 55% [62], and COVID-19 rates are around 20%[63]. The persistent sector of the population that remains under-vaccinated for diseases is troublesome [64]. Games like N.O.V.E.L. may be able to play a role in shifting public perception.

Given that the students who played the game more strongly agree that human testing ensures that new vaccines are safe to use, that vaccines don’t generally contain harmful substances, and that even healthy people should get vaccinated, we feel that the game can be a tool in the broader efforts for public education in two ways. First, the students recognized the need for a broader conversation about vaccines, openly discussing with their teachers about their experience with COVID vaccine hesitancy among their parents and community. While further study would be needed, it would be interesting to explore whether students with their new knowledge and confidence would become trusted ambassadors of that knowledge and have positive impacts on vaccination rates by sharing within their family and community [36]. Consider that 88% of Americans believe that the childhood MMR vaccine is beneficial, yet there is a decline in the number of adults who support the vaccination requirements for school age children [58]. Furthermore, some adults change their perspective and become vaccine hesitant due to lack of trust and concerns over safety [65]. As such, a second opportunity for public education could come from the students’ developing their own strong knowledge base and positive attitudes toward vaccines, which is reinforced and carried with them as they mature into adulthood.

Additional benefits of playing N.O.V.E.L. include that students were able to transfer the pro-social soft skills needed to successfully win the game to a recognition that these same skills are needed to be a good scientist. The game also illustrates that scientific discovery is a process [66], not usually an “a-ha!” moment. This is important because the percent of U.S. adults who have confidence in scientists was at 86% in 2019, but dropped to 73% in 2023 [67]. A learning activity that creates positive reinforcement for trust in scientists and their positive role in society would be a welcome benefit of playing the game.

Limitations

Classroom implementation strategies varied across study sites due to the unique settings, constraints, and considerations of each individual classroom environment. Factors such as schedule limitations, classroom culture, and educator preferences influenced how the N.O.V.E.L. was facilitated. Researchers were intentionally not present during implementation in order to minimize the potential for the Hawthorne effect [68] and to allow for more authentic integration of the materials into each classroom’s existing routines. Limitations of this assessment also include that it is not a longitudinal study. Further, that we rely on teachers’ reporting that playing the game was effective for students from diverse backgrounds. Taken together, repeated exposure would likely enhance student understanding, which is why we have expanded the collection of complementing teacher resources and aligned curriculum for a multi-dimensional learning engagement [2].

Conclusions

This study represents the first use of a cooperative, table-top board game designed and used for advancing a change in attitudes towards vaccine knowledge and vaccine hesitancy. The game format shifted adolescent attention away from computer screens or traditional classroom lectures to a focused group interactive play and conversation among peers. Their aim is to cooperatively work for a common goal—to make a vaccine to save the world. This shift in student attention away from screens was seen as a significant positive that teachers reported as engaging students who were generally not actively interested in normal classroom activities. Through this gameplay, students gained knowledge about vaccines and gained confidence in their knowledge. The students also recognized key skills that are important for being a scientist including being cooperative, creative, sharing resource and being a good communicator. Overall, our evaluation of N.O.V.E.L. has demonstrated the promise of gamified learning for critical community health issues that can occur while having fun.

Illustrative case study

We highlight the experiences of one teacher and her classes to demonstrate N.O.V.E.L.’s immediate impact. While Stacie’s reflections echo those of the other teachers, her students’ demographics were quite different. Stacie teaches in a rural region affected by economic disinvestment. Her students’ experience with the game illustrates the value of gamified vaccine learning in a medically underserved community.

Context

Stacie is a science teacher with more than 24 years of experience. At the Title 1, Lake Middle School, she teaches approximately 250 students (7th and 8th grade, ages 12– 14). The school is predominantly white (99.6%, Table S1) and located in a rural area where the nearest college is over forty-five miles away.

Although there was no publicly available information to determine the number of students with disabilities, Stacie shared that nearly a third of her students had an Individualized Education Plan (IEP) and she teaches half of the school’s student population with diagnosed disabilities. Most of her students do not have behavior disorders but rather live with learning disabilities. She refers to her students with disabilities, whether they are neurodivergent and/or have various physical attributes, as “exceptional learners.” She stated:

“Many of my students come from poverty, live with grandparents, and struggle with getting basic needs.… Next year, it is estimated that my school will have about 80 exceptional learners, so 32%. I will more than likely teach over half of the 80 exceptional learners.”

Lake Middle School is in one of the poorest counties in the U.S.. The community has historically relied on the coal industry. However, since coal’s value has dropped, many working-class families in her county are unemployed and living with a median annual household income of $28,000. Stacie explained, “Our community subsists on the coal and logging industries, the local hospital, and the local school board. Jobs are not diverse, nor are they plentiful.” Stacie said she wanted to give her students new opportunities, and that she would chase after anything she could find to support her students and their families.

Student participants

Originally, Stacie was recruited to participate in the N.O.V.E.L. study through a university health and science director in her state. She asked several of her classes to be part of the N.O.V.E.L. study, ultimately providing data from 104 consented students (completing 100 pre-gameplay surveys, 95 post-gameply surveys, and 91 matched before/after gameplay surveys). This was the largest number of student participants from one teacher in the collective dataset. As instructed, students were placed into groups of four at six tables per class. Stacie said because there were five sections of the game, they took five days to play it. She said students were able to complete this work in her 48-min class periods.

Student impacts

N.O.V.E.L. was a welcome alternative to Stacie’s more didactic, state-mandated curriculum.

“It gave my students an avenue to where they really had to communicate with one another and they had to use critical thinking skills…we get so tired at teaching to a test instead of letting these kids do some trial and error, letting them experiment, giving them opportunities to communicate, so I think it was a really good opportunity for my kids.”

“We test kids way too much. We don't give them time to really understand, to really learn. We focus more on quantity rather than quality. And that's a detriment to our students. You’re never gonna raise any scores unless you give them the time to really explore and learn.”

She witnessed students’ cooperation and communication in action as they shared resources and strategies or taught each other scientific vocabulary.

“Sometimes they wouldn’t have a lot of [in-game] money, so as a group, they would donate for the kids who didn’t have a lot and they would make suggestions: ‘Well, I could donate to you, and then that way it would free us up to do the next part to get to the next step’.”

“With Phase 5—the distribution- they really enjoyed it. They were kind of comparing it to Pictionary—how now you can move or rotate cards and you can swap cards out. They liked that and it got really strategic then because they understood. They said ‘If I swap this card for that one, it may cause a problem in another area.’ They really got to the point where they would collaborate and make a decision before somebody moved any pieces”

Consistent with these observations, Stacie’s students were able to transfer their experience to the ideas that sharing resources, being communicative, and being cooperative were more important for real-world scientists (Supplement Fig. CS1A, see Supplement Table S15 for statistics).

Stacie observed her students’ engagement throughout the gameplay: “I didn’t have any kids that were, like, clowning around, and I didn’t have anybody out in space. They were really into it, and they got really competitive with it.” She even told her supervisor when he visited her classroom during gameplay, “You have to learn by not just being on that computer.” She shared that the supervisor commented, “They're really playing this. And they're really into it.”

Stacie thought N.O.V.E.L. should be offered in schools, especially ones with limited resources. She noted the game was especially beneficial for students with disabilities.

“My students were excited and challenged to learn about infectious diseases through playing the board game. It allowed my students to learn visually and kinesthetically. My exceptional learners learned more through the experience than they would have learned any other way. Most importantly, my students had fun learning.”

“[In Phase 2, my exceptional learners] really liked how things were color coded. It was easier for them to understand, even when we were in phase one with the pathogen code and when they sat down and we were discovering which one of the pathogens they had.”

We did not collect data about whether students had a disability, so we could not disaggregate our findings along these lines. However, Stacie’s students gained about five points (22%) on the vaccine development items (Supplement Fig. CS1B), a large, statistically significant increase (t(90) = 8.47, p < 0.001, Cohen’s d = 0.89).

N.O.V.E.L. inspired students to talk about vaccine hesitancy in their families and community.

“We got a conversation going on about how we've came through COVID, and how things go, and for a lot of my kids, maybe their parents didn't take them to get the vaccine. They got into a conversation about all the different information and how they would hear about that vaccine, and why people wouldn't take it versus…they were made to take it because they had to work. We got into a little bit of ethics with it.”

Students’ own beliefs about vaccine safety and efficacy changed very little (Supplement Fig. CS1C and Supplement Table S16). None of the before/after gameplay differences were statistically significant using a threshold of 0.008 (alpha of 0.05 divided by 6 comparisons, calculated to protect against finding false positives). However, the increase in the item about human testing was significant at the 0.05 level (t(90) = 1.74, p = 0.04, Cohen’s d = 0.18).

Stacie’s students also learned about human subjects’ research and informed consent through the process of following the study’s protocols. They asked, “Why do we have to take this [consent form] home, get permission from our parents?” The study gave Stacie the opportunity to talk more about research and the “scientific method.” She elaborated with her students as follows:

“…when you do research and you're underage, you have to have parental consent, even though it sounds silly that you're playing a game. It's part of the research protocol. And so then they started asking some questions. They opened their eyes a bit on it and [it] gave them an idea about the scientific method.”

Teacher benefits

Not only did Stacie’s students learn from N.O.V.E.L., but she felt that “it was a good one for me, because I got to see them in a different light. It was my opportunity… to see them learn as they would go.” She said it can be easy for teachers to feel they are not supported or do not have the resources to continue teaching. The game provided an opportunity for teachers and students to engage in the learning process alongside their students. Stacie wants to continue learning because if she doesn’t “use it,” she will “lose it.”

“I feel like sometimes I stagnate in in my teaching, so every year I look for something new to do. I've had some really great opportunities in various fields to bring things into my classroom. But I really like the concept that was in this game. And you're gonna learn about viruses and how things are developed for vaccines, and it is completely different. [the alternative is that] I'm gonna put [the students] on a computer. And I'm gonna give [them] a research topic, but that’s just stuff that's boring. It's not interactive, and this game is.”

“You have to give [the students] new things to do, and it doesn't only help them, but it helps me as a teacher, too.… So, this was something that was really important. They need to know about the viruses, my goodness, after COVID, of course; and they need to know about vaccines, because so much of the misinformation that was put out. So this really helped open their eyes a bit.”

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

Conceptualization: BK, JAP Methodology: BK, KB, KMB, JAP Investigation: BK, KB, KF, KMB, JAP Visualization: KF, KMB, JAP Funding acquisition: JAP Project administration: BK, JAP Supervision: BK, KMB, JAP Writing—original draft: BK, KF, KMB, JAP Writing—review & editing: BK, JAP All authors reviewed the manuscript.

Funding

The project was supported by funding from the National Institutes of Health, National Institute of General Medical Sciences through a Science Education Partnership Award (SEPA) Supplement 3R25GM132910-05S1. Additional support was provided by Duquesne University School of Education and School of Science and Engineering, and the Grable Foundation.

Data availability

All data are available in the main text or the supplementary materials.

Declarations

Ethics approval and consent to participate

Institutional Review Board (IRB) approval included that for all research involving human subjects, freely-given written informed consent to participate in the study and written consent to publish would be obtained from participants and that for children under the age of 18, their parent or legal guardian provided written consent and that the child provided assent to participate and to publish. After receiving IRB approval (Heartland IRB file number HIRB Project No. 10052023-515; Clinical Trial Number—not applicable) the review of the above referenced protocol was conducted in accordance with the ethical principles of the Belmont Report. Throughout, the research was carried out following the guidelines of the ethics committees listed in this statement and written informed consent for participation and publication was obtained from those who involved in the current study.

Informed consent

The IRB approvals included specific consent to participate for all participants over the age of 18 years and that for children under the age of 18, their parent or legal guardian provided written consent and that the child also provided assent to participate.

Consent for publication

The IRB approvals included specific consent for data publication for all participants over the age of 18 years and that for children under the age of 18, their parent or legal guardian provided written consent and that the child also provided assent for data publication.

Competing interests

The authors declare no competing interests.