Co-designing and pilot testing a digital game to improve vaccine attitudes and misinformation resistance in Ghana

ABSTRACT

Misinformation related to vaccines has been shown to potentially negatively impact public perceptions and intentions to vaccinate in many contexts including COVID-19 vaccination in Ghana. Psychological inoculation – where recipients are warned about the misleading techniques used in misinformation – is a potential intervention which could preemptively boost public resistance against misinformation. Cranky Uncle Vaccine is an interactive, digital game that applies inoculation, offering a scalable tool building public resilience against vaccine misinformation and promoting positive health-related behaviors. In this study, we document the process of developing and testing a West African version of Cranky Uncle Vaccine, with co-design workshops and a pilot test conducted in urban and peri-urban areas of the Greater Accra region of Ghana with 829 young people who had access to mobile and computer devices. The average age was 21.8 and participants were highly educated (median education level “Some/all university”) with slightly more females (51.2%) than males (48.4%). Pilot participants filled out surveys before and after playing the game, measuring vaccine attitudes (pre-game M = 3.4, post-game M = 3.6), intent to get vaccinated (pre-game M = 3.5, post-game M = 3.6), and discernment between vaccine facts and fallacies (pre-game AUC = 0.72, post-game AUC = 0.75). We observed a significant improvement in attitudes toward vaccines, with players demonstrating increased likelihood to get vaccinated after completing the game. Among players who indicated that they were unlikely to get vaccinated in the pre-game survey (N = 52, or 6.3% of participants), just over half of these participants (53%) switched to likely to get vaccinated after playing the game. Perceived reliability of vaccine facts remained the same, while perceived reliability of vaccine fallacies significantly decreased, indicating improved ability to spot misleading arguments about vaccines. These results demonstrate the effectiveness of a digital game in building public resilience against vaccine misinformation as well as improving vaccine attitudes and intent to get vaccinated.

Introduction

Misinformation has become a global phenomenon in recent years and has serious consequences for public health. Health workers have been grappling with the challenge of combating vaccine-related misinformation for years,¹ and the recent COVID-19 pandemic witnessed the emergence of numerous strategies aimed at misinforming the public about the rationale behind COVID-19 vaccines.² Misinformation has been associated with decreased intention to vaccinate in many contexts, including before and during the COVID-19 vaccination campaigns in Sub-Saharan Africa and Ghana.^3–6

Significant efforts are being made to combat vaccine misinformation, including initiatives such as sensitization and awareness programs, and public health education. Digital interventions, such as online games, have been suggested as a means to mitigate misinformation among the public.⁷ One such intervention, the Cranky Uncle Vaccine game, was originally co-designed and tested in East Africa.⁸ This paper documents the subsequent co-design process of Cranky Uncle Vaccine in Ghana, making the game contextually relevant to the Ghanaian context and target audiences (community health workers and young people, aged 16–30), and the challenges that have emerged from its piloting in Ghana. We also report results from a pilot study conducted in Ghana, testing the game’s effectiveness in improving vaccine attitudes and discernment between vaccine facts and misinformation.

Inoculation theory

A substantial body of cognitive psychology research underscores the enduring impact of misinformation. Correcting falsehoods is challenging once they have taken root in memory,⁹ and repeated exposure tends to enhance the perceived accuracy of fake news.¹⁰ As a result, some scholars have begun to explore the prospect of proactive strategies to counteract the spread of misinformation.¹¹ Given the striking resemblance between the propagation of fake news in online networks and the replication of a virus,¹² one promising approach has been the resurgence of inoculation theory.

Inoculation theory draws an analogy from the field of medicine, where protection against future viral threats is achieved by exposing individuals to weakened forms of the virus. Similarly, attitudinal inoculation aims to protect individuals from future persuasive threats by exposing them to weakened forms of argumentation.¹³ This theory has been applied across various domains, including politics and health, over the course of more than 50 years. The theory relies on two primary mechanisms: (a) forewarnings or threats of counter-attitudinal attacks to stimulate resistance and (b) preemptive refutation of these attacks to guide the process of counterarguing and equip individuals with specific content to counter future persuasive challenges.^11,14 Although meta-analyses have demonstrated the effectiveness of inoculation messages,¹⁵ early inoculation research primarily focused on “cultural truisms,” which are beliefs so widely accepted within a social context that the idea of persuasive attacks against them seemed unlikely.¹³ Notably, one of the latest applications of inoculation theory pertains to addressing contested science, misinformation, and conspiracy theories.¹⁶

Furthermore, current research on inoculation theory encounters two primary limitations. First, scholarly attention has predominantly concentrated on establishing resistance to specific issues.¹⁷ Second, the traditional approach to preemptive refutation has been more passive – where recipients passively receive an inoculation message – rather than active – where recipients actively interact with the inoculation intervention.¹⁸ These limitations significantly constrain the scalability and applicability of the “vaccine” metaphor.⁷ Consequently, recent investigations have shifted focus toward the concept of a “broad-spectrum vaccine” against misinformation⁷ that focuses on the rhetorical techniques (technique-based) or logical fallacies (logic-based) found in misinformation.

The broad-spectrum approach introduces two theoretical innovations. Firstly, it shifts the emphasis away from proactively exposing participants to weakened instances of specific (mis)information and instead exposes them to weakened examples of the techniques commonly used to produce most misinformation. Secondly, it revisits McGuire’s initial prediction¹⁹ that active inoculation, involving participants in generating their own “antibodies,” may be more effective in building resistance to persuasion than the passive provision of a defensive pre-treatment.

In a novel paradigm introduced by Roozenbeek and van der Linden⁷ participants engaged in a simulated social media environment where they encountered diluted forms of misinformation strategies and are actively encouraged to create their own content. This intervention takes the form of a free social impact game called “Bad News” (getbadnews.com), developed in collaboration with the Dutch media platform DROG. In this game, players learn about six common misinformation techniques: impersonation, emotional language, group polarization, spreading conspiracy theories, discrediting opponents, and trolling. The game’s objective is to craft and propagate disinformation within a controlled environment while gaining an online following and maintaining credibility. The theoretical rationale behind incorporating these six strategies is elaborated upon in Roozenbeek and van der Linden⁷ and addresses many prevalent disinformation scenarios, such as false amplification and echo chambers. Although the game scenarios are fictitious, they draw inspiration from real-world events. In essence, this gamified approach to inoculation theory incorporates an active and experiential element into the process of building resistance.

Misinformation games

Over the years, gamification – the use of game mechanics in non-gaming contexts²⁰—have been recognized as powerful tools for enhancing concentration and learning. In the educational field and various other disciplines, gamification has been employed to improve academic performance by harnessing the motivational aspects of games. This educational approach is termed “gamification,” with a focus on its ability to solve problems or achieve organizational goals.²¹ Gamification is described as the incorporation of key elements of games, such as fun, rewards, and competition, into non-game settings, such as website design or application development, to enhance user engagement.²² Numerous studies have conducted experiments that consistently demonstrate the positive impact of gamification on both psychological and physical behavior changes.²³ The incorporation of gamification in systems can effectively foster self-awareness,²⁴ boost user engagement, and evoke feelings of excitement, enjoyment, and motivation while encouraging improvements in specific behaviors.²⁵

In a case study conducted by Park and Kim,²⁶ they explored the effectiveness of a gamified educational curriculum, which included elements like points and leaderboards, in increasing student participation in a classroom setting. An expanding body of research has focused on the development of gamified systems aimed at enhancing awareness of cybersecurity and preventing future cyberattacks. For instance, Canova et al.²⁷ introduced “NoPhish,” a gamified mobile app designed to heighten users’ self-awareness in identifying phishing links.

Gamified inoculation offers an effective way to boost engagement and can be readily adapted to incorporate new technology, tackle emerging false claims, and respond proactively to impending threats.¹⁸ When a game is enjoyable, individuals are more inclined to participate in it and share their experiences, thereby enhancing its effectiveness. This increased engagement gives gamified approaches a better chance of combatting the “virus” of false claims when compared to traditional fact-checking methods.¹¹

While games tailored to specific lessons and contexts typically yield superior results, broadly applicable games can also initiate valuable discussions.²⁸ This flexibility is particularly valuable for addressing complex or confusing subjects. Incorporating real-world propaganda and disinformation techniques into games provides individuals with an opportunity to practice recognizing deceptive or inaccurate claims in a low-risk environment.²⁹ Furthermore, using fictional scenarios, rather than real-world examples, allows learners to explore sources and ideas without the burden of preexisting knowledge about specific claims or concerns about providing incorrect answers. The lower risk associated with fictional games can boost engagement by encouraging learners to take educated risks without fearing negative consequences on their actual beliefs or knowledge.¹⁷

Repetition of claims represents another crucial element integrated into gamified approaches. Repeating an unfamiliar claim tends to increase both belief in that claim and the retention of specific details.³⁰ The illusory truth effect, caused by repeated exposure to misinformation along with other factors, can significantly sway individuals toward accepting false claims.¹² As a result, both accurate and inaccurate statements are intentionally repeated in the game to mirror the repetition observed in social media and among those intentionally spreading disinformation. Research on other games has also indicated that extended playtime leads to better retention, suggesting that a repetitive or extended approach can be advantageous.³¹ In the gamified classroom environment, repetition can counter the recurrence of real-world false claims.

Cranky Uncle Vaccine

The Cranky Uncle Vaccine game was adapted from the existing digital game Cranky Uncle (crankyuncle.com), which was launched in 2020 on smartphones and browsers.³² Both games combine logic-based and active inoculation and cartoon humor to build resilience against misinformation. The game incorporates the two elements of an inoculation intervention by 1) including a threat warning that misinformation about vaccines casts doubt on facts about vaccines, and 2) providing counter-arguments by explaining the misleading rhetorical techniques Cranky Uncle used to cast doubt on scientific facts. The techniques were taken from the FLICC taxonomy, which documents the five categories of science denial techniques (fake experts, logical fallacies, impossible expectations, cherry picking, and conspiracy theories) and a number of rhetorical techniques, fallacies, and traits of conspiratorial thinking under these categories.³³ The Cranky Uncle Vaccine game focused on the 10 most common fallacies found in vaccine misinformation (see Appendix A1 for detailed descriptions and literature for each fallacy, and Figure 1 for an example of an explanation of a fallacy) Eight of these were found in the original Cranky Uncle game, with two new techniques added (false cause, natural is best). The primary target user groups for the game were young people (aged 16–30) and community health workers, country-wide in Ghana.

Figure 1.

Screens showing an explanation of a misinformation technique. a) shows the healthcare worker explaining facts about vaccines and vaccine research (fact-based inoculation). b) and c) show cranky uncle explaining a misinformation technique he uses to cast doubt on vaccine facts.

The game’s content, including the fallacy explanations, game play, and cartoon characters, were refined in a series of co-design workshops held in Uganda, Kenya, and Rwanda.⁸ This process resulted in a single version of the game developed for the three East African countries, which was subsequently pilot-tested in Uganda and Kenya.³⁴ Pilot participants for the Uganda/Kenya pilot were recruited through local partner organizations, targeting urban and rural populations. Participants were predominantly younger (average age 27 years) and well educated (median education level “some/all college”). The game significantly improved vaccine attitudes, including intent to get vaccinated, and improved players’ ability to distinguish true from false statements. Cranky Uncle Vaccine was made publicly available at crankyunclevaccine.org, with the smartphone version of the game available in several African countries including Ghana and the browser version available globally.

This study documents the development of a Ghana version of the Cranky Uncle Vaccine game through a series of co-design workshops conducted in Ghana. We also report the results of a pilot test of the game conducted with Ghanaian participants.

Materials and methods

Co-design workshop

Co-design, also known as collaborative or co-creation design, is a methodology that involves active participation from various stakeholders, including designers, developers, end-users, and other relevant individuals, in the design and development process of a software product.³⁵ To refine content for the Ghana version of the game, a number of co-creation workshops were organized in the Volta and the Greater Accra regions with young people, health workers, and health promotion officers from those two regions. In the Volta Region, separate sessions were held with young people (aged 17–32 years old) and health workers in the Central Tongu district (Figure 2). In the Greater Accra Region, two separate sessions were held with health workers and young people selected from Ayawaso West Municipality.

Figure 2.

Co-creation session with health workers in the Central District, Volta Region.

During the various sessions, the East African version of the game was presented to the participants through printed out copies, scenario cards, and feedback questionnaires. Participants worked through the game’s content in small groups by reading and discussing each scenario card representing common misinformation tricks before returning to the plenary session for discussions and feedback collection (see Figure 2). This offered participants the opportunity to digest and assimilate the content in order to identify keywords, characters, costumes, and expressions that were alien to Ghanaians. At the end of the co-creation sessions, draft content was pre-tested with health promotion officers, community members, and the youth to validate the content and the key characters in the game.

Feedback from the workshops were collated to inform visual design of the game’s characters. Sketch variations and final character designs are shown in Table 1. Workshop participants asked that the Cranky Uncle character be amended to have round glasses and a walking stick, with a bright-patterned shirt featuring a pocket and pens. Participants recommended the health worker wear a standard nurse uniform with a name badge. For the older woman, a print was added to her head scarf and clothes, as well as jewelry worn in the local style. The younger characters were dressed in modern clothes and accessories such as headphones and watches. The secondary characters (older woman, younger woman, younger man) were used in the quiz questions (examples shown in Figure 3).

Table 1.

Initial sketches through to final, amalgamated design of the Ghanaian cranky uncle cartoon characters.

Character	Option 1	Option 2	Option 3	Final Design	Revisions from Co-design workshops
Cranky Uncle					Slightly plumper, with gray, balding hair. Shirt to have a bright pattern, no collar and be untucked. Add a pocket and pens. Add round glasses and a walking stick. Shoes to be a flat slip-on style.
Health Worker/Nurse					A standard nurse uniform (knee-length green dress with round white collar and white apron). Hair to be up under a standard nurse cap. Shoes are a flat slipper style. Remove the stethoscope and add a name badge.
Older woman					Add a print to the head scarf. Add gold hoop earrings and a necklace made of local beads. Add a large woven basket. Clothing to have a print, with shorter puff sleeves. Shoes to be a flat slipper style.
Younger woman					She should be wearing distressed jeans, and a sleeveless top. Hairstyle to be longer braids. Add makeup and gold hoop earrings. Add a small watch and a small, cross-body handbag.
Younger man					Hoodie should have a centered lower pocket and be unzipped, with a t-shirt underneath. Change shoes to sneakers. Hair should be a close shave on the sides with short twists or dreads. Add headphones and a watch.
Professional with a disability					This additional character with a disability was requested and approved by UNICEF.

Figure 3.

Selected quiz questions using different supporting characters to give examples of fallacies: a) false cause fallacy–with the young woman and young man characters, b) personal stories fallacy–with the older woman and cranky uncle characters, c) natural is best fallacy.

Pilot test

Recruitment

In order to evaluate how effectively the game could be used to assess vaccine-related behaviors in Ghana, Prof. Emmanuel Awuni Kolog of the University of Ghana conducted a pilot study on a sample size of 829 participants, composed of university students and health workers in Ghana. The sample size was determined through power analysis, assuming a small effect size on some dependent variables as shown in the Uganda/Kenya pilot study. Participants provided informed consent via the research form embedded within the game, then filled out the pre-game survey (median time 3.4 minutes) which measured attitudes toward vaccines and collected demographic data. After completion of the pre-game survey, participants were guided to play the game. Players who completed the 10 fallacy explanations (median time spent 41 min) were then invited to fill out the post-game survey (median time 3.0 min) again evaluating their attitudes toward vaccines and requesting feedback on the game.

Measures and methods

The measures used in this experiment and described below were the same as those used in³⁴ with the exception of one of the fallacy measures (conspiracy theory) being excluded due to a technical error during data collection.

Methods

To explore whether participants showed significant improvement on dependent variables over the course of the game, paired t-tests were conducted comparing pre-game measures to post-game measures. ROC (Receiver Operating Characteristic) analysis was also employed with the area under the curve (AUC) acting as a measure of discernment between factual and false statements. The significance level used was p < .05.

Demographics

In the pre-game research survey, participants were asked demographic questions concerning age, gender, region, and education level.

Vaccine attitudes

Participants answered three questions, each on a four-point scale. The first item asked whether the participant was personally for or against vaccination (from “strongly for vaccination” to “strongly against vaccination). The second item measured their agreement with the statement “I feel that it is important that I get vaccinated” (from “strongly agree to strongly disagree). The third item queried participants’ likelihood to get recommended vaccinations for themselves or their children (from “Very likely” to “Very unlikely”). The first two items were drawn from the validated Vaccination Trust Indicator.³⁶

Fact reliability

Participants were asked to rate the truthfulness of two factual statements about vaccines on a four-point scale (from “True” to “False”). A four-point scale was used rather than binary classification to capture that participants often had degrees of agreement with vaccine statements, and allowed the calculation of ROC curves as a measure of discernment. These items were chosen to reflect the broad themes expressed in the fact-based content in the game. The factual statements were “Vaccines are among the most thoroughly tested medical products available” and “Vaccines help protect us against many serious diseases that can pass from person to person.”

Fallacy reliability

Participants rated the truthfulness of five misleading statements, each representing a different fallacy: natural is best (“Natural herbs boost the immune system better than vaccines”), false cause (“Someone got sick after a vaccine so the vaccine must be to blame”), evil intent (“The COVID-19 virus was intentionally created and spread for profit reasons”), pick and choose (“Vaccines aren’t safe as a study was published mentioning safety concerns”), and personal attack (“Drug companies earn large amounts of money from vaccines so vaccines are untrustworthy”). The fallacy items used the same 4-point scale as the two fact items. The fallacy items were identified as common arguments from a literature review of research studying vaccine misinformation.⁸

Game feedback

Participants who completed the game and the post-game survey answered two open-ended questions soliciting qualitative feedback about the game: “What stood out the most to you in the game?” and “What changes would you like made to the game?” Answers to both questions were content analyzed by two annotators, using the major themes identified in Cook et al.³⁴ Interrater reliability was calculated using Gwet’s AC1 given the imbalanced categories, with some themes more highly represented than others.³⁷ Interrater reliability for the “What stood out” answers showed substantial agreement, with a Gwet’s AC1 of .79 (95% CI: .76, .82), while answers for “what changes would you like made” showed very good agreement, with a Gwet’s AC1 of .92 (95% CI: .9, .94). To resolve discrepancies, the two annotators deliberated and decided on consensus codings in cases of disagreement.

Results

Demographic questions concerned age (Mean M = 21.8, Standard Deviation SD = 5.5), gender (51.23% female, 48.4% male), and education level (median education level “Some/all university”). The wording of the questions are given in Appendix A2 and a table of demographic data is provided in Appendix A3.

Main effects

Figure 4 shows the pre-game and post-game vaccine attitudes. Paired t-tests showed significant improvement from pre-game to post-game in general vaccine attitude (p < .001, d = 0.3), vaccine importance (p < .001, d = 0.18), and intent to get vaccinated (p = .005, d = 0.14). While a small number of participants (n = 52 or 6.2%) expressed vaccine hesitancy in the pre-game survey, selecting unlikely or very unlikely to get vaccinated, just over half of these participants (n = 28 or 53%) switched to likely or very likely to get vaccinated in the post-game survey.

Figure 4.

Vaccine attitudes before (red) and after (blue) playing the game, measuring whether participants are generally for or against vaccines (pre-game M = 3.4, SD = 0.7; post-game M = 3.6, SD = 0.7; p < .001), perceived importance of vaccines (pre-game M = 3.5, SD = 0.7; post-game M = 3.6, SD = 0.6; p < .001), and likelihood of getting recommended vaccinations (pre-game M = 3.5, SD = 0.7; post-game M = 3.6, SD = 0.7; p = .005).

Signal detection theory offers a technique for exploring potential changes in players’ discernment between factual and misinforming statements.³⁸ ROC analysis offers a way to examine discrimination between accurate and false information, with the area under the curve (AUC) of a ROC curve being a single measure of discrimination. Figure 5 shows the ROC curve summarizing our participants’ discrimination between facts and misinformation. The paired samples t-test showed a statistically significant increase from pre-game AUC values (M = 0.72, SD = 0.28) to post-game AUC values (M = 0.75, SD = 0.32, p < .001, d = 0.121). The Bayes factor indicated moderate evidence in favor of the alternative hypothesis, BF = 17.5.

Figure 5.

ROC (Receiver Operating Characteristic) curves for the pre-game and post-game surveys.

Figure 6 shows the pre-game and post-game perceived reliability of vaccine facts and fallacies (see Appendix A2 for wording of research questions). In paired t-tests, participants showed an improvement in perceived reliability for fact #1 (p = .003, d = 0.1). However, they showed a decrease in the perceived reliability of fact #2 (p = .01, d = 0.09). Participants showed a significant decrease in their reliability ratings for the natural is best fallacy (p < .001, d = 0.41), false cause (p < .001, d = 0.36), evil intent (p < .001, d = 0.22), personal attack (p < .001, d = 0.13), and pick and choose (p < .01, d = 0.09).

Figure 6.

Perceived reliability of vaccine facts: fact 1 (pre-game M = 3.5, SD = 0.7; post-game M = 3.6, SD = 0.7; p = .003), fact 2 (pre-game M = 3.6, SD = 0.7; post-game M = 3.5, SD = 0.8; p = .01), and vaccine fallacies: natural is best (pre-game M = 2.9, SD = 1.0; post-game M = 2.4, SD = 1.1; p < .001), false cause (pre-game M = 2.4, SD = 1.1; post-game M = 2.0, SD = 1.1; p < .001), evil intent (pre-game M = 2.2, SD = 1.1; post-game M = 1.9, SD=1.1; p < .001), pick & choose (pre-game M = 2.1, SD = 1.0; post-game M = 2.0, SD = 1.1; p = .01), and personal attack (pre-game M = 2.0, SD = 1.0; post-game M = 1.9, SD = 1.1; p < .001). Red represents pre-game measures, blue represents post-game measures.

Content analysis of player feedback

Player feedback included highlight themes and requested changes. Table 2 lists the highlight themes in order of prevalence, with example quotes from each category. The importance or benefits of vaccines stood out most to players (32.3%), consistent with the Uganda/Kenya pilot studies in.³⁴ The second most standout feature of the game was the misinformation tricks (26.3%). Some participants reported a transformative effect on their understanding of vaccines with comments such as “I misunderstand what vaccination ought to be. But now I have learnt and understood it” and “I have all my doubts on vaccines cleared now.” Strikingly, one participant confessed “that at some point I believed I was a cranky uncle applying” false cause” just because two things happened coincidentally. There are a lot of uncle cranky’s among us and I did not realize until I took the quiz.”

Table 2.

Themes identified in response to the question “what stood out the most to you in the game?,” percentage of participants in each theme, and examples.

Highlight Theme	Percent	Examples
Importance & benefits of vaccines	32.3	“There is a better chance of being safe from a disease through vaccination than natural medicine,” “Vaccinations are very important since it prevent one from contracting diseases,” “Vaccines are safe and helps fight against certain disease that get transmitted through physical contact”
Tricks used to mislead	26.3	“How we mostly have a lot of tricks to talk ourselves out of being vaccinated,” “Spreading false rumors is a disease itself,” “The tricks made me see the vaccinations in a different light,” “That I shouldn’t misrepresent”
Educational value and knowledge gain	9.4	“How you are able to learn so much in such a short time and the graphics too were cool,” “I think the game is a great way to create awareness about the importance of vaccines,” “The fact that I got to learn all that just in one game is very amazing. I have all my doubts on vaccines cleared now,” “The questions and the anecdotes were very well thought out”
Engagement and game experience	8.2	“The quizzing techniques employed to ensure participants understand what they are into,” “Every level makes me smarter,” “How the game has been structured with the levels,” “The level of interaction”
Misinformation & misconceptions	7.2	“I was surprised that people think the vaccine create a magnetic field on the body”
Other	7.1	“Nothing,” “Everything,” “Covid 19”
Characters	6.5	“Cranky uncle was very manipulative,” “Cranky uncle’s frown,” “Cranky uncle’s unwillingness to change his opinion,” “How cranky uncle thoroughly pointed out most of the things I think about vaccines,” “The crankier cranky uncle gets,” “The opinions of the uncle and how they are familiar,” “The nurse was very strict on Uncle’s claims and always confident on the scientific proves about vaccination.”
Skepticism and doubts about vaccines	3.0	“Both sides of the argument are convincing. You don’t know which side to support,” “It appeared this was not a game, but an attempt to ‘educate’ people on vaccines,” “Natural or traditional remedies help boost immune system better than vaccine,” “Herbal medicines are better than vaccines”

Participants also reported finding the real-world examples of misinformation instructive, commenting on “how relatable the situations are” and highlighting “the use of scenarios to explain every trick (it made my understanding of the tricks easier).” Participants reported enjoying different aspects of the game such as the interactive quizzes, commenting that “the quizzes made it easy to comprehend everything.” The gameplay elements also had the effect of incentivizing players to go further into the game, highlighting “the urge to continue with the quiz and to know more.” Given the visual nature of the game, some participants focused on the visual aspects of the characters, focusing on “the cranky old man’s expressions” and commenting that “the expressiveness of the characters was very interesting.” Some participants highlighted “Cranky uncle’s unwillingness to change his opinion,” a feature we will return to when discussing requested changes. Participants also focused on the nurse character and her “efforts to rebut cranky uncle’s claims.” However, the response from participants wasn’t universally positive with 3% of feedback expressing vaccine hesitant opinions such as “vaccines can’t be trusted.” Similarly, some expressed a negative opinion of the game such as “the game pushes the ideas it wants to.”

Table 3 shows the most prevalent requested changes. By far, the most prevalent change was “None” with the majority of players indicating satisfaction with the game (62.5%). While some players criticized the amount of content in the game, some preferred that there be less content (7.1%) while others indicated they would have liked more content (5.4%). When asked what changes they would like made, the majority of participants requested no changes, with comments such as “No changes the game is perfect and very educational” and “I enjoyed every bit of it.” Several participants expressed desire for a version of the game more suitable for younger audiences, with comments such as “there should be more of public education to the youths” and “I hope they create one for kids.” A recurring theme in this pilot study’s feedback as well as the previous pilots was requesting audio of the characters talking, “to help people who can’t read.”

Table 3.

Themes identified in response to the question “what changes would you like made to the game?,” percentage of participants in each theme, and examples.

Theme	Percent	Examples
No changes	62.5	“It was just perfect to me,” “None. The game was amazing,” “Nothing. It was very educational and straight to the point. Also easy to navigate,” “The game is actually good. It’s perfect”
Other	10	“Hmm there should be more of public education to the youths,” “I will encourage more people to get involve,” “In my opinion, the game really good but I hope they create one for kids,” “Is should be announced to the local community and also to be demonstrated to them,” “Maybe a cranky auntie”
Gameplay	8.9	“Faster response,” “The bandwidth should be increased a little,” “It should be fast,” “I would probably modify the options available to the users to make them less obvious to select,” “The quiz should be a bit more difficult,” “Some of cranky uncle’s thinking must change,” “The cranky uncle should trust in vaccines,” “To change uncle crunky mind-set,” “The rules should be clearer,” “There should be a high score to be attained in order to complete the game”
Less content	7.1	“It’s sooo long,” “Reduce the tricks,” “The game takes a longer period to finish and the levels should be reduced,” “The time involved must be decreased,” “Too many questions”
More content	5.4	“A little more quizzes should be added. All the same it is still good,” “I don’t think it needs any changes. Just some more tricks.,” “More evidence,” “More levels or games should be added to it,” “More stories should be created to defend the vaccine”
Add sound	2.5	“Add a little of voice to the images to help people who can’t read,” “I would like to hear the characters talking. It would be more fun and educational,” “I think there should be a read-out-loud option,” “Make the people talk instead of us reading,” “Maybe add some game sounds to make it more lively”
More/different visuals	2.0	“Maybe a bit more interactive and seeing Crank uncle get mad,” “More characters should be incorporated,” “Some facial expressions of characters should match results”
Simplify language	1.3	“It’s somewhat confusing at some point,” “Less complex.” “Medical words should be made simple for understanding.” “Some words are quite ‘huge.’ Simpler words could be used”
Cultural inclusion	0.1	“Making it more accessible to spheres of people by considering persons with Disabilities,” “The game should also talk much about how to encourage people in the rural area much more than the cities”

Discussion

Ghana, like most African countries, was inundated with disinformation during the COVID-19 pandemic. A number of studies suggested that this was potentially affecting people’s attitudes and intentions, leading the Ghana Health Services (GHS) and UNICEF to establish a systematic infodemic management process during the pandemic to address misinformation related to COVID-19 vaccines.³⁹ This system informed communication responses to specific misinformation narratives. Here, building upon this pioneering infodemic management work already established in Ghana, we describe the development and testing of a preemptive intervention to build public resistance to misinformation.

This study reaffirmed results from the pilot studies conducted in Uganda and Kenya,³⁴ finding that among Ghana participants, the Cranky Uncle Vaccine game increased vaccine attitudes and discernment between facts and misinformation. As previous pilot studies showed the effectiveness of the game in only two East African countries, these results show that the game is also effective in a West African country. More broadly, this adds to the growing body of evidence finding the effectiveness of digital games countering misinformation. Other studies have found misinformation games have been effective in improving players’ ability to identify misinformation statements.^7,17

However, there has been mixed results in whether other games have been successful in not only reducing belief in misinformation but also increasing acceptance of factual information.³⁸ Our study also showed mixed results in this regard, finding a decrease in agreement with the second fact. There are several possible explanations for this result. One possibility is a ceiling effect with pre-game values for Fact 2 being slightly higher than Fact 1. A more likely possibility is an ordering effect in the survey design with the fact 2 question positioned in the middle of the fallacy questions (see Appendix A2 for the question order). Possible experimental designs should randomize the question order to eliminate ordering effects.

The qualitative feedback collected from the post-game survey was broadly consistent with feedback collected from previous pilots.³⁴ The importance and benefits of vaccines was the feature that stood out most to players, with the tricks used to mislead being the second-most prominent feature. The feedback of participants reporting a transformative effect from the game was consistent with the quantitative result of 53% of participants who were unlikely to get vaccinated before playing the game switching to being likely to get vaccinated. We also note that this result is consistent with the findings from the Uganda/Kenya pilot, where 58% of participants who were unlikely to get vaccinated switched to likely after playing the game.³⁴ Another notable piece of feedback on what participants would like changed was a request for audio of characters talking. It would be worth exploring whether this feature could be automated (to avoid having to rely on voice actors for each region that the game is released in).

We made one methodological change to the content analysis in this study relative to³⁴ adding a new theme “Gameplay,” which covered a variety of requested changes to the game. A few participants reported that the game was slow, commenting that “The responses were a bit slow. For example it took like 4 or 5 seconds for the next button to actually move to the next page. Could be done a little faster.” The slowness in the game’s response may have been due to the fact that during the pilot, data was retrieved from a server based in the U.S. rather than a server that was geographically closer. Some participants complained “that the answers are easy to spot,” suggesting that more difficult questions might be a good addition, particularly later in the game. Another interesting request was that Cranky Uncle should change his mind-set from the start to the end of the game, indicating players’ desire to see the character become more accepting of vaccines by the end of the game. Some participants also commented that “there should have been accurate instructions on what to do.” While the game does include an onboarding sequence explaining the game (see Figure 7), these instructions could be fleshed out to make the mechanics of the game more clear to players from the start.

Figure 7.

Onboarding script.

As acknowledged in³⁴ the field-test nature of this pilot study means there are some limitations to this study. The convenience sampling approach meant that the participants were younger and higher educated than the average population. The pre-game and post-game surveys were deliberately designed to be short to facilitate game play, but that meant a number of key measures were omitted from the research design. One important measure in inoculation theory is threat, as one of the mechanisms by which inoculation interventions are effective is by raising people’s perceived threat from persuasive messages.¹⁴ Another key measure in inoculation interventions is counterarguments, with inoculating messages stimulating people’s ability to raise counterarguments against persuasive messages.⁴⁰ Measuring both threat and counterarguments are beyond the scope of this study given the limited duration of the in-game surveys, but other experiments conducting surveys outside the game may explore these concepts in greater detail.

One of the most encouraging results from this pilot was the increase in vaccine intent, especially among those expressing vaccine hesitancy before the game. However, we do recognize that the vaccine intent question asked how likely the participant was to get vaccinations “for yourself or your children,” so we do not distinguish between people vaccinating themselves and parents vaccinating their children. Measuring the impact of the game on these two distinct intentions is a topic for future studies.

The content from the Cranky Uncle Vaccine game has been converted to other formats and released in Ghana. A WhatsApp chatbot version of the Cranky Uncle Vaccine game was developed in collaboration with Ghana-based team of global mobile-based intervention company Viamo. For the chatbot game, the content was simplified with the ten misinformation tricks streamlined down to seven techniques. In addition, an Interactive Voice Response (IVR) version was also developed and translated into five local Ghanaian dialects (Dagabni, Ewe, Ga, Hausa, Twi), with the IVR game being made available on the UNICEF Agoo mobile platform. Measuring the effectiveness of these two platforms is beyond the scope of this study and a topic for future study.

To conclude, this study has added to previous research finding that the Cranky Uncle Vaccine game is effective across multiple African countries in improving vaccine attitudes and intent, as well as increasing players’ ability to discern misinformation. Further, we find our intervention is effective in the Global South, helping to address the issue that over 80% of research into misinformation interventions to date has been conducted in the Global North.⁴¹ It remains to be seen whether our results in the specific communities studied generalize to find that the game is effective in other languages, on other platforms, and in other countries that are culturally different to the countries already tested.⁴² Nevertheless, replication of the positive impact of the game across different countries adds to research finding that inoculation is an effective tool against misinformation and digital games offer a scalable means to reach large and diverse populations.

Biography

John Cook is a Senior Research Fellow with the Melbourne Centre for Behaviour Change at the University of Melbourne. He researches how to use critical thinking to counter misinformation. In 2007, he founded Skeptical Science, a website that won the 2011 Australia Museum Eureka Prize for the Advancement of Climate Change Knowledge. In 2013, he published an award-winning paper quantifying the 97% scientific consensus on climate change which was highlighted by President Obama and UK Prime Minister David Cameron. He developed the Cranky Uncle and Cranky Uncle Vaccine games, combining critical thinking, cartoons, and gamification to build resilience against misinformation. He works with organizations such as Facebook, NASA, the Sabin Vaccine Institute, and UNICEF to develop evidence-based responses to misinformation.

Appendix.

Fallacy/Trick	Description
Appeal to nature (Natural is best)	The view that something natural is inherently good while unnatural things are inherently bad.43–45 In the context of vaccines, this fallacy takes the form of assuming that because vaccines are man-made, they are unnatural and therefore potentially dangerous or inferior to natural immunity.46,47
Post hoc (False cause)	Latin for “after this, therefore because of this,” this fallacy confuses correlation with causation. It involves incorrectly identifying two things as being causally associated without enough evidence to do so.47,48 A common example is the claimed link between autism and the MMR vaccine.47
Evil intent	Suspicion about individuals, organizations, corporations, or overarching belief systems (e.g., doctors, the government, pharmaceutical companies, ‘Western medicine’) are an integral feature of anti-vaxxers and conspiracy theorists.49 Conspiracy theorists consider healthcare and government systems untrustworthy because they believe they are corrupt and colluding with pharmaceutical companies, have conflicts of interest, and are deceiving the people.46
Anecdote (Personal stories)	This fallacy prioritizes personal experiences over scientific evidence, referring to first-hand ‘testimonies’ and personal ‘narratives’ as ‘evidence’ that vaccines are injurious and harmful.46 An example is a heart-breaking story from a mother about her child being hospitalized shortly after a vaccination.49
Ad Hominem (Personal attack)	Translated from Latin for “to the person,” ad hominems attempt to discredit a person’s arguments or science by personally attacking them.33 One type of ad hominem is genetic fallacy, where arguments are dismissed by their source of origin. For example, the fact that vaccines are made by pharmaceutical companies is enough to discredit them.43
Misrepresentation	Generally, the misrepresentation fallacy involves misrepresenting a situation or system in such a way as to distort scientific understanding. For example, the claim that vaccines can cause the diseases they are meant to prevent, or that vaccines contain active viruses.46
Cherry picking (Pick and choose)	This fallacy involves focusing on individual cases or data that seem to confirm a particular position, while ignoring a significant portion of related cases or data that may contradict that position.43
Conspiracy theories	Vaccine conspiracy theories involve governments, pharmaceutical companies, doctors, CDC/WHO, or the media, conspiring to deceive the public about vaccine dangers/adverse side effects from the public (Fasce et al., 2021). Conspiracy theories are an integral feature of the anti-vaccine movement.43
Impossible expectations	Unrealistic standards of safety or effectiveness are often demanded when it comes to vaccine safety (Stolle et al., 2020). This involves the demand that vaccination should be 100% safe, and because absolute safety cannot be promised, vaccines are flawed and dangerous.46
Fake experts	People are more likely to rely on ideas offered by expert sources but often lack the resources, knowledge, or time to resolve whether someone is an expert or not. This makes them vulnerable to the influence of “fake” experts, who represent themselves as possessing relevant knowledge and expertise when they have none.50 Appealing to fake expertise is also known as an argument from false authority.43

Appendix.

• Q1. Can you share your age in years? (pre-game only)

  • Text box
• Q2. Please share your gender. (pre-game only)

  • Options: Female, Male, Other, Prefer not to say
• Q3. Please share your country in which you live and if you can, the county where you currently live. (pre-game only)

  • Text box
• Q4. Please share your highest level of education completed. (pre-game only)

  • No formal education
  • Some/all of primary school
  • Some of secondary school/high school
  • Completed secondary school/high school
  • Some/all college/diploma level
  • Some/all university/degree level
  • Some/all postgraduate degree level
• Q5. Thinking about vaccination in general, would you say you are personally …

  • Strongly for vaccination
  • Somewhat for vaccination
  • Somewhat against vaccination
  • Strongly against vaccination
• Q6. Please respond to the following statement: “I feel that it is important that I get vaccinated.”

  • Strongly agree
  • Somewhat agree
  • Somewhat disagree
  • Strongly disagree
• Q7. How likely are you to get recommended vaccinations for yourself or your children, if applicable?

  • Very likely
  • Somewhat likely
  • Somewhat unlikely
  • Very unlikely
• How true do you think the following statements are?

  • Q8 (Fact #1). Vaccines are among the most thoroughly tested medical products available.
  • Q9 (Natural is best). Natural herbs boost the immune system better than vaccines.
  • Q10 (False cause). Someone got sick after a vaccine so the vaccine must be to blame.
  • Q11 (Evil intent). The COVID-19 virus was intentionally created and spread for profit reasons.
  • Q12 (Fact #2). Vaccines help protect us against many serious diseases that can pass from person to person.
  • Q14 (Pick & choose). Vaccines aren’t safe as a study was published mentioning safety concerns.
  • Q15 (Personal attack). Drug companies earn large amounts of money from vaccines so vaccines are untrustworthy.
  • Options for Q8 to Q15:

    • True
    • Mostly true
    • Mostly false
    • False
• Q16. What stood out the most to you in the game? (post-game only)

  • Text box
• Q17. What changes would you like made to the game? (post-game only)

  • Text box

Note: due to a technical issue, the following question was not included in our dataset:

• Q13 (Conspiracy theory). The COVID-19 vaccine contains a microchip to track people.

Appendix.

Table A1.

Demographics of pilot participants.

Demographics	Description	Number	Percentage
Age	18-20	493	59.5
	21-30	270	32.6
	31-40	53	6.4
	41-50	9	1.1
	51-60	2	0.2
	61-70	2	0.2
Gender	Female	424	51.1
	Male	402	48.5
	Other	2	.2
	Prefer not to say	1	.1
Education	No formal education	0	0.0
	Some/all of primary	2	0.2
	Some of secondary	11	1.3
	Completed secondary	263	31.7
	Some/all college	41	4.9
	Some/all university	471	56.8
	Some/all postgraduate	41	4.9

Funding Statement

This project received funding from Global Affairs Canada under the grant ‘Canada’s Global Initiative for Vaccine Equity (CanGIVE)’, and funding from UNICEF Ghana for the piloting in Ghana.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Author contributions

JC, CL, AT, EAK, II and KLH contributed to research design and ethics application; and WC contributed to the content design and image creation. JC and WC content analyzed player feedback. EAK contributed to pilot study management, recruitment of study participants, and data collection. JC and EKA wrote the introduction and the literature review. JC conducted data analysis. CL, AT, KLH contributed to the interpretation of the results. JC led the first draft of the manuscript; and CL, AT contributed to reviewing and further revisions, writing of the manuscript. II and SB led the preparation and recruitment of participants for co-creation workshops and contributed to the writing of abstract and other sub-themes of the paper. All authors provided final approval of the paper prior to submission.

Dava availability

The datasets generated during and/or analyzed during the current study are available in the OSF repository, https://osf.io/rvuzf/?view_only=a45a8ff792124ef0b0b7c00beef27cd2.

Ethical approval

Ethical approval was obtained from the University of Ghana Ethics Committee for Humanities (reference number ECH 168/22-23).