Fair Play: A Study of Scientific Workforce Trainers’ Experience Playing an Educational Video Game about Racial Bias

Capitalizing on the potential effectiveness of game-based learning to have a positive impact on persistent subtle race bias in academic science, the authors developed Fair Play, an interactive role-playing educational video game targeting science faculty. Fair Play appears to promote perspective taking and bias literacy, which can reduce race bias.

Abstract

Explicit racial bias has decreased in the United States, but racial stereotypes still exist and conspire in multiple ways to perpetuate the underparticipation of Blacks in science careers. Capitalizing on the potential effectiveness of role-playing video games to promote the type of active learning required to increase awareness of and reduce subtle racial bias, we developed the video game Fair Play, in which players take on the role of Jamal, a Black male graduate student in science, who experiences discrimination in his PhD program. We describe a mixed-methods evaluation of the experience of scientific workforce trainers who played Fair Play at the National Institutes of Health Division of Training Workforce Development and Diversity program directors’ meeting in 2013 (n = 47; 76% female, n = 34; 53% nonwhite, n = 26). The evaluation findings suggest that Fair Play can promote perspective taking and increase bias literacy, which are steps toward reducing racial bias and affording Blacks equal opportunities to excel in science.

INTRODUCTION

Despite more than 40 years of concerted efforts to increase the participation of racial and ethnic minorities in higher education (Kaplin and Lee, 2013), Blacks remain disproportionately underrepresented as students and faculty in science fields (Nelson and Rogers, 2003; Nelson, 2007; National Science Board, 2012). A diverse scientific workforce is needed to drive innovation and to keep the United States a global leader in science and technology (National Academy of Sciences, National Academy of Engineering, Institute of Medicine, 2007). Thus, the persistent underparticipation of Blacks in science is a costly waste of talent the nation cannot afford (National Academy of Sciences, National Academy of Engineering, Institute of Medicine, 2007).

A large body of theoretically supported experimental and observational research describes how racial stereotypes impede the participation and advancement of Blacks in academic fields like science (Devine, 1989; Steele, 1997; Dovidio et al., 2002; Bertrand and Mullainathan, 2004; Biernat, 2012). Stereotypes portray Blacks as deficient in the skills and traits associated with science aptitude (Steele and Aronson, 1995; Steele, 1997; National Academy of Sciences, National Academy of Engineering, Institute of Medicine, 2007; Biernat, 2012; Muhs et al., 2012; National Research Council [NRC], 2013). For example, in one study, “scientific” topped the list of traits viewed as most uncharacteristic of Blacks (Rothbart and John, 1993). This “lack of fit” for Blacks in science influences individual Black students, as well as their peers, mentors, and evaluators, to assume that Blacks lack the competence to succeed in science careers (Steele and Aronson, 1995; Steele, 1997; Biernat and Kobrynowicz, 1997; Biernat et al., 2009; Biernat, 2012; Hodson et al., 2002; Reid and Radhakrishnan, 2003; Swim et al., 2003; Bertrand and Mullainathan, 2004; Harper and Hurtado, 2007; National Academy of Sciences, National Academy of Engineering, Institute of Medicine, 2007; Smith et al., 2007; Sue et al., 2008; Sue, 2010; McCabe, 2009; Block et al., 2012; Muhs et al., 2012; NRC, 2013). As a result, Blacks face disadvantages in the personal interactions and review processes that are critical for cultivating interest in science and achieving professional success (Steele and Aronson, 1995; Steele, 1997; Biernat and Kobrynowicz, 1997; Biernat et al., 2009; Biernat, 2012; Hodson et al., 2002; Reid and Radhakrishnan, 2003; Swim et al., 2003; Bertrand and Mullainathan, 2004; Harper and Hurtado, 2007; National Academy of Sciences, National Academy of Engineering, Institute of Medicine, 2007; Smith et al., 2007; Sue et al., 2008; Sue, 2010; McCabe, 2009; Block et al., 2012; Muhs et al., 2012; NRC, 2013).

Contemporary research on prejudice demonstrates that stereotypes can influence reasoning, decision making, and evaluative judgments at both explicit (i.e., conscious, intentional) and implicit (i.e., unconscious, unintentional) levels (Devine, 1989; Dovidio et al., 2002). Although explicit forms of racial bias (e.g., unambiguous exclusion of Blacks from schools or programs; Kaplin and Lee, 2013) have become less common (Ancis et al., 2000; Pololi et al., 2010; Ong et al., 2011), subtle forms of racial bias, such as implicit bias, remain largely unaddressed in academic science. Such subtle bias is problematic, because most people are unaware of it and its impact on judgment (Devine, 1989; Dovidio et al., 2002; Nosek et al., 2007; Greenwald et al., 2009), and because it can influence reasoning and decision making, even among those who consciously hold egalitarian beliefs (Devine, 1989). Experimental studies show that discrimination from subtle bias, however inadvertent and unintentional, can take many forms: it can present as peers, colleagues, and evaluators holding Blacks to higher standards to confirm their job competence (e.g., by needing to show a greater number of skills or accomplishments; i.e., “competency proving”; Biernat and Kobrynowicz, 1997); attributing their accomplishments to others or to luck (i.e., “attributional rationalization”; Blair et al., 2013; Block et al., 2012; Haynes and Heilman, 2013); mistaking them for other members of their racial group (i.e., “failure to differentiate”; Ackerman et al., 2006; Hugenberg et al., 2010; Brebner et al., 2011); assuming they hold a low- (or lower) status position (i.e., “status leveling”; Smith, 1985); or targeting subtle remarks or actions at them that communicate discriminatory messages (i.e., “microaggressions”; Sue, 2010). Subtle bias conspires in multiple ways to constrain opportunities for Blacks and may help to explain results from studies that find Black students and faculty feel less supported, are offered fewer resources, and perceive a more negative campus climate than whites (Ancis et al., 2000; Reid and Radhakrishnan, 2003; Pololi et al., 2010, 2013; Ong et al., 2011).

Subtle racial bias is also linked to disparities in performance expectations and outcomes. For example, Amodio and Devine (2006) found that those with stronger implicit race stereotypes expected poorer performance of a Black than an academically equivalent white student on math and verbal tests. Other studies show that subtle racial stereotypes can lead equivalently qualified Black students to underperform academically relative to white students—a phenomenon called “stereotype threat,” in which anxiety about confirming a negative stereotype about the lack of competence of one’s group causes cognitive distraction that impedes performance (Steele and Aronson, 1995; Steele, 1997). Stereotype-based bias can additionally influence evaluators to rate identical achievements from Black and white individuals differently (Steele and Aronson, 1995; Steele, 1997; Biernat and Kobrynowicz, 1997; Biernat et al., 2009; Biernat, 2012; Hodson et al., 2002; Reid and Radhakrishnan, 2003; Swim et al., 2003; Bertrand and Mullainathan, 2004; Harper and Hurtado, 2007; National Academy of Sciences, National Academy of Engineering, Institute of Medicine, 2007; Smith et al., 2007; Sue et al., 2008; Sue, 2010; McCabe, 2009; Block et al., 2012; Muhs et al., 2012; NRC, 2013). Importantly, the cumulative impact of subtle bias over the course of an academic career contributes to the high rates of attrition observed for racial and ethnic minorities from science—a loss that perpetuates demographic disparities in the scientific workforce and impedes diversity of thought and innovation in science fields (National Academy of Sciences, National Academy of Engineering, Institute of Medicine, 2007; Valantine and Collins, 2015).

Interventions show promise for reducing racial bias in evaluative judgments and personal interactions. In particular, perspective taking, which involves inviting participants to imagine and write about the life and experience of a Black individual viewed in a photo or video, is effective for reducing implicit racial bias and addressing negative intergroup behavior, because it increases empathy for Blacks (Batson et al., 1997; Todd et al., 2011). Another technique, promoting bias literacy—defined as learning a vocabulary about bias and a skill set to recognize, understand, converse about, and intentionally practice cognitive and behavioral strategies to mitigate the impact of group stereotypes on judgment and decision making (Sevo and Chubin, 2008)—has been used to successfully reduce implicit racial bias in students (Devine et al., 2012) and to reduce biased behavior and improve department climate for faculty (Carnes et al., 2012, 2015) in academic science fields. Such educational interventions approach stereotype-based bias as a “habit of mind” that can be changed just like any other unwanted habit (Devine et al., 2012; Carnes et al., 2012, 2015).

Despite proof of effectiveness, widespread dissemination of bias-reducing interventions across academic institutions has not been feasible because of factors such as cost, complexity, and lack of trained facilitators. In recognition of the complex strategies needed to address scientific workforce diversity at a national level, the federal government launched several initiatives calling for the development and use of innovative methods to bolster interest in science and increase the participation of underrepresented minorities in science careers (U.S. Office of Science and Technology Policy, 2011, 2014; Robinson et al., 2014; Valantine and Collins, 2015). One area of focus involves using video games for teaching and learning (U.S. Office of Science and Technology Policy, 2013, 2014). We posited that an educational video game involving role-playing would be an effective intervention tool to address racial bias in science, because it would be easy to disseminate and would allow players to engage in perspective taking and become “bias literate” through experiential learning (Gee, 2007; Gutierrez, Kaatz, et al., 2014). This premise is supported by contemporary theory in game-based learning that avatar-based video games promote perspective taking and domain literacy by inducing a state called projective identity (Gee, 2007).

In game play, projective identity arises as a consequence of a player’s dependence on an avatar character to succeed and is characterized as the merging of a player’s and avatar’s identities (Gee, 2007). As players persevere in games, their projective identity grows stronger, allowing them to increasingly feel or suffer what their avatar experiences. This suggests that role-playing in video games allows players to develop empathy for an avatar character (Gee, 2007; Gutierrez, Kaatz, et al., 2014). Because success in a game depends on the player’s ability to learn and apply new knowledge and skills relevant to the game environment, projective identity also necessitates and facilitates the development of literacy about the game domain (Gee, 2007). By inviting players to take the perspective of a racial/ethnic minority avatar in science who experiences bias, we theorized that game play could promote empathy for the minority character’s racial group and help players to become “bias literate” as a means to foster the reduction of implicit racial bias (Batson et al., 1997; Gee, 2007; Sevo and Chubin, 2008; Todd et al., 2011; Devine et al., 2012; Carnes et al., 2012, 2015).

Recognizing the multiple ways in which game-based learning could have a positive impact on racial bias in academic science, we developed Fair Play, an interactive, role-playing, educational video game targeting science faculty. We targeted faculty because their interaction with students is pivotal for increasing interest and persistence in science careers (Pfund et al., 2014). In Fair Play, players assume the role of Jamal Davis, a Black student working toward a graduate degree in science at a large research university. We selected this name because it is associated with Black males in the United States (Bertrand and Mullainathan, 2004). As Jamal, players experience and learn about implicit bias as they interact with other game characters and complete tasks relevant to a science graduate student, such as selecting an advisor, writing research articles, and attending professional conferences.

Earlier versions of the game (Table 1), tested on science graduate students at the University of Wisconsin–Madison, showed that 75% (n = 94/125) of players were able to take Jamal’s perspective and that players who reported high empathy for Jamal showed lower implicit racial bias scores, as measured on the Implicit Association Test (Gutierrez, Kaatz, et al., 2014). Despite these results, this early version of the game was problematic, because bias scenarios were not well integrated into the story line and game mechanics were not easily grasped by nongamers. This earlier version of the game was also tested on science graduate students and not faculty. In this paper, we describe an evaluation of the game completed by a national sample of scientific workforce trainers. This group included faculty, many of whom had knowledge of diversity issues.

TABLE 1.

Previous versions and descriptions of game prototypes, 2011–2013

Version	Game components	Role of player	Purpose of game
1	Lab Dash (directing a science lab)	Play as Lab Director: hire staff; submit papers; and address bias events	To show how time pressure and high cognitive load can increase the influence of implicit bias on decision makinga
2	Lab Dash: an almanac to introduce concepts; and interactions with NPCs (nonplayable characters)	Jamal or Geoffrey: play first as a Black and then as a white graduate student	To promote perspective taking and to develop bias literacy by having players experience implicit bias as Jamal, but not as Geoffrey
3	Different environments (e.g., lab, conferences); SciConnect—networking tool; new NPCs; and continued use of bias almanac	Jamal: interact with NPCs while in graduate school; succeed in networking despite bias incidents	Perspective-taking, as Jamal; increased friendliness or respect from NPCs leading to upgrades in the lab; to promote bias literacy, each NPC associated with one implicit bias concept
4	Fair Play: a point-and-click, avatar-based role-playing game; use of different environments, NPCs, and bias almanac	Jamal Davis: a Black student working to matriculate in graduate school despite experiencing subtle racial bias	To build player bias literacy through: perspective taking, as Jamal, while experiencing bias encounters in the environment or with characters; naming biases stored in almanac; exposure to counter stereotypic exemplars and images

^aMartell, 1991; Dijker and Koomen, 1996; Perry et al., 1996; Sczesny and Kuhnen, 2004; Wigboldus et al., 2004.

METHODS

All facets of this study were approved by University of Wisconsin–Madison’s Institutional Review Board for Human Subjects Research.

The Game

The design team consisted of game developers from Games + Learning + Society (www.gameslearningsociety.org) and experts in science career development and stereotype-based bias from the Women in Science and Engineering Leadership Institute (http://wiseli.engr.wisc.edu/) and the Center for Women’s Health Research (www.womenshealth.wisc.edu/Home.aspx). At weekly meetings over a 3-year development period, our team used the iterative process of rapid-cycle prototyping for game development (Figure 1; Squire and Jenkins, 2003; Squire, 2006; Gee, 2007; Fullerton, 2008; Brathwaite and Schreiber, 2009; Kapp et al., 2013; Schell, 2014). In this process, game design centers on a topical focus, but content, characters, and a story line emerge through development of rough prototypes that are piloted, assessed, and revised. We assembled informal groups of students, faculty, and staff to provide feedback during multiple cycles of development and revision. We produced several nonplayable and four playable versions of the game that were ultimately crafted into the fifth and final version (Table 1).

FIGURE 1.

Rapid cycle prototyping for game development in Fair Play.

In the final version of Fair Play, players are explicitly introduced to Jamal as he begins graduate school. Players experience realistic scenarios that a graduate student would encounter in pursuit of a doctoral degree in science. In chapter 1, Jamal must identify an advisor, secure funding, and write a personal statement, all while experiencing subtle racial bias. Subsequent chapters become increasingly complex with regard to the academic tasks Jamal must complete (e.g., requiring players to work with faculty members, write papers, and attend academic conferences), and the types of biases he encounters (Figure 2). To bolster engagement, players have agency to choose whom they encounter in their path through the game with consequences that impact players’ experiences. Biases are classified as “critical,” “environmental,” and “conversational.” Critical biases are on the “critical path,” which means a player will always encounter them while playing through the game; all other biases are optional and may not be experienced by every player (Table 2). Each time a bias encounter occurs, players must identify the type of bias they experience in an almanac, which remains available to them “on demand” (Figure 3). To select the bias concepts players encounter in Fair Play, our team combined a broad, integrative literature review with weekly discussions. We converged on a final list of bias concepts that fit within the story line of the game and provide a balanced overview of the types of biases racial/ethnic minorities commonly experience in academic fields like science. Table 2 contains a complete list of bias concepts, definitions, in-game examples, and references.

FIGURE 2.

Jamal experiencing an environmental racial microaggression in the final version of Fair Play.

TABLE 2.

Bias concepts, definitions, and location in Fair Play^a

Bias construct	Definition	In-game examples	References
Attributional rationalization	Group stereotypes may lead to assumptions that people from underrepresented groups are less competent than their majority peers. As a result, they may not receive credit for their accomplishments, which are often incorrectly attributed to those in the majority or to factors other than their efforts (e.g., luck).	Chapter 3: Environmental bias Bias encounter: You overheard people talking about Deirdre’s tenure case, dismissing her contributions to an article. Bias hint: People are talking about Deirdre’s tenure case. Chapter 5: Critical bias Bias encounter: Lucas unfairly presumed that your high-quality proposal would be attributed to him rather than you, perhaps because of group stereotypes about ability. Bias hint: A graduate student talks about your proposal.	Block et al., 2012; Blair et al., 2013; Haynes and Heilman, 2013
Color-blind racial attitudes	Color-blind racial attitudes reflect the belief that discrimination no longer exists. Though based on the positive premise that we should all be treated equally, a color-blind approach discounts the experiences of members of minority groups and can backfire by promoting bias.	Chapter 2: Critical bias Bias encounter: Nick mentioned that paying attention to race is unimportant and that it has nothing to do with graduate studies. Bias hint: A faculty member talks about departmental fit. Chapter 2: Environmental bias Bias encounter: You overheard two staff members outside Nick’s office talk about how they do not see race any longer; they only see people. Bias hint: Staff members are chatting outside Nick’s office. Chapter 5: Critical bias Bias encounter: Lucas mentioned that students of color are unfairly advantaged by diversity programs that target underrepresented minorities because he assumed discrimination no longer exists. Bias hint: A graduate student believes students of color are unfairly advantaged by diversity programs.	Ryan et al., 2007; Plaut et al., 2009; Morrison et al., 2010
Competency proving	To counter common assumptions about their presumed incompetence, members of minority groups frequently and repeatedly have to demonstrate that they are indeed qualified, capable, and/or competent.	Chapter 1: Critical bias Bias encounter: Morgan suggested that instead of assuming you are here because of your academic accomplishments, people may assume you are only attending the university to play basketball. Bias hint: A researcher discusses assumptions about your admission to the university. Chapter 4: Critical bias Bias encounter: Nick was surprised by the quality of your excellent paper, which he had asked you to write to prove your academic ability. Bias hint: A professor is surprised by your academic writing ability.	Edwards, 2000; Ginther et al., 2011; Amodio and Devine, 2012
Failure to differentiate	Members of a particular minority group may sometimes be mistaken for one another by a person of a different group. All groups share this unintentional recognition bias, but research suggests the effect is most pronounced for white individuals when viewing racial and ethnic minorities.	Chapter 1: Conversational bias Bias encounter: Shania confused you for another Black graduate student she had met at a research lab. Bias hint: A graduate student is hiding on the balcony to get work done. Chapter 2: Critical bias Bias encounter: Shania confused you for another Black graduate student she works with in the office. Bias hint: A graduate student is waiting for Tyrone.	Ackerman et al., 2006; Hugenberg et al., 2010; Brebner et al., 2011
Impression management	People from historically low-status or underrepresented groups must often pay more conscious attention to how they behave (e.g., a Black student may consciously modulate his/her tone of voice or volume of speech to prevent activating the racial stereotype of being angry or aggressive) or how they dress in order to reinforce their professional roles. A casual appearance may elicit prevailing negative images of their group.	Chapter 5: Environmental bias Bias encounter: You dressed up for your presentation, and you were glad that your formal clothing made people recognize you as a conference participant instead of as support staff. Bias hint: Check to make sure you are well-dressed for the conference.	Morris et al., 1996; Harrison, 2002; Harlow, 2003; Dowling, 2011; McDermott and Pettijohn, 2011
Racial microaggression	Microaggressions are brief and subtle comments, behaviors, or environmental cues that intentionally or unintentionally communicate hostile, derogatory, or unwelcoming messages toward members of underrepresented groups. When accumulated, these seemingly minor messages lead to harmful isolation and alienation. There are three types of microaggressions: microassaults, microinsults, and microinvalidations.	Chapter 2: Environmental bias Bias encounter: Portraits of the past faculty in the department showed that they were all older white men. This is an example of a microinvalidation. Bias hint: Portraits hanging in the hallway show the past faculty in the department. Chapter 3: Critical bias Bias encounter: Franklin singled you out and asked you, but not others, for ID before you could enter the library. This is an example of a microinsult. Bias hint: A library staff member asks to see your ID. Chapter 3: Critical bias Bias encounter: Engaging in racial profiling, Franklin followed you around the stacks. This is an example of a microassault. Bias hint: A library staff member keeps a close eye on patrons’ activities. Chapter 3: Environmental bias Bias encounter: The book collection on Black history was separated from American history in the stacks. This is an example of a microinvalidation. Bias hint: There are perplexing disciplinary divisions among the library’s collections.	Sue et al., 2007, 2008; Sue, 2010; McCabe, 2009
Shifting standards of judgment	The presumed incompetence of members of underrepresented groups causes well-qualified, underrepresented individuals to be judged as highly competent when compared with members of their group. But, they are held to even higher standards and require greater proof of competence than comparable members of the majority group.	Chapter 1: Critical bias Bias encounter: Lucas suggested that you apply for fellowships targeted for minorities, because he presumed that you have lower competence compared with all applicants but would be considered highly competent when compared with other minority applicants. Bias hint: A graduate student gives you advice for funding. Chapter 5: Critical bias Bias encounter: Lucas presumed there are higher expectations for his performance and is ignorant of how evaluators actually require greater proof of minority group members’ ability before confirming their competence. Bias Hint: A graduate student presumes there are higher expectations for his performance.	Biernat et al., 2009, 2010
Status leveling	Based on stereotypes about the lower social standing of minority groups, status leveling occurs when a person from an underrepresented group is assumed to belong to a lower social category or position.	Chapter 1: Critical bias Bias encounter: Lucas assumed that you were a caterer for the incoming students’ event and asked you to get more soda. Bias hint: A graduate student notices that the buffet is low on soda. Chapter 3: Critical bias Bias encounter: A tutor mistook you for a student who needed tutoring. Bias hint: A tutor cannot find the person he is looking for.	Smith, 1985
Stereotype threat	Stereotype threat occurs when awareness of negative stereotypes about one’s own group induces stress and anxiety about confirming the stereotype. Situations that consciously or unconsciously trigger stereotype threat can lead members of minority groups to underperform relative to their actual ability.	Chapter 5: Conversational bias Bias encounter: When Crystal found out she was the only female presenter on a panel, she became nervous, and it compromised her performance. Bias hint: A fellow student shares her experience of being a minority at a previous conference.	Steele and Aronson, 1995; Steele, 1997; Fries-Britt and Griffin, 2007; Smith et al., 2007;
Tokenism	Tokenism is treating members of minority groups as representative of their entire group rather than as individuals, especially when they are a numeric minority or the only person from that group present (solo status).	Chapter 1: Critical bias Bias encounter: Morgan wondered whether the lack of minorities makes the campus less appealing to other Black students and asked for your opinion. Bias hint: A researcher is concerned with the lack of minorities on campus. Chapter 2: Critical bias Bias encounter: Morgan asked you to speak to his class about the Black graduate student experience. Bias hint: A researcher asks you to speak to his class. Chapter 4: Critical bias Bias encounter: Nick suggested that you may have been admitted to the university through a diversity program as a token minority. Bias hint: A professor discusses your admission to the university. Chapter 5: Critical bias Bias encounter: Lucas implied that indicating a fake minority status would advantage him in an application process. Bias hint: A graduate student discusses the application process.	Kanter, 1977; Smith, 1985; Niemann, 1999

^aThere are three types of biases: critical, environmental, and conversational. Critical: biases on the “critical path,” which means a player will always encounter them on the first play-through of the game; all other biases are optional and may not be experienced by every player. Critical biases are typically conversational biases, which occur through conversations with other characters in the game. Environmental: biases present in the environment through interacting with objects in the world or observing ambient conversations between NPCs. Conversational: biases that occur through conversations with other characters in the game. Not every conversation in the game contains a bias nor is every conversational bias also a critical bias.

FIGURE 3.

Example of a bias concept (i.e., competency proving) and its definition from the bias almanac in the final version of Fair Play.

To offer players the opportunity to engage in more diverse styles of learning, additional instructions are provided through interaction with nonplayable characters (NPCs) regarding the goals of the game and how to navigate the point-and-click actions. Visual cues such as directional arrows guide players in moving Jamal through the story line; stars above their heads indicate NPCs with whom Jamal must interact; and red exclamation marks and a sound alert occur when Jamal encounters bias. Players are also guided by being provided information about the tasks they must complete in each of five chapters.

As players become more bias literate and comfortable in navigating the game, their knowledge and ability are continually tested and expanded. Incorrectly identifying bias concepts in the almanac impacts players’ trajectories, particularly at the end of the game, when correctly identifying biases is necessary for entry to the final chapter. Thus, to succeed in the game, players must become bias literate—meaning that they must learn how to recognize, label, and talk about bias as they engage in perspective taking as Jamal.

Study Design

We performed a postplay, mixed-methods survey of a convenience sample of scientific workforce trainers to assess their experiences playing the first two chapters of Fair Play.

Participants

Trainers were attendees of the National Institutes of Health (NIH) Division of Training Workforce Development and Diversity (TWD) program directors’ meeting for fiscal year 2013. This group was predominantly involved in projects funded by the National Institute of General Medical Sciences (NIGMS) aimed at broadening the participation of historically underrepresented group members in science—and included faculty principal investigators (PIs), and program directors/administrators or staff at the NIH or a recipient institution. Of the 47 participants, 76% were female (n = 34); 47% were white (n = 21); 24% were Black (n = 11); 13% were Hispanic/Latino (n = 6); 12% were Asian (n = 5); 4% were Native American/Alaskan Natives (n = 2); and 4% were of unknown race/ethnicity (n = 2). Most participants were program directors/administrators (56%, n = 24) or faculty PIs (41%, n = 18).

Data Collection

Computer stations were set up for 2 days of the TWD conference to allow any attendee to play the game (with assistance, if requested). After playing the game (approximate length of time was 20–30 minutes), participants read an informed-consent information sheet and voluntarily completed an online mixed-methods survey (i.e., “session survey”) that probed their experiences and asked them to evaluate specific aspects of the game. Quantitative survey questions asked players to rate their level of agreement on a five-point scale (where 1 = strongly disagree and 5 = strongly agree) that they 1) found it easy to take Jamal’s perspective, 2) understood how Jamal felt during bias incidents, 3) enjoyed playing the game, 4) occasionally felt uncomfortable playing the game, 5) thought the game accurately portrayed racial bias, 6) thought bias incidents in the game happen to Blacks in academic settings, 7) thought the game was an effective way to teach about bias, 8) would play this game again in the future, and 9) would recommend the game to a colleague. They also responded yes or no regarding whether they 10) could see a use for the game in their professional settings or work (Figure 4; Appendix A in the Supplemental Material). Open-ended qualitative questions asked participants to 1) use their own words to describe the objective of Fair Play, 2) describe what it is like to be Jamal in an academic setting, 3) provide comments or explanations for any of the quantitative questions, and 4) explain how they could use the game in their professional settings or work if they indicated they would do so (Appendix A in the Supplemental Material).

FIGURE 4.

Flow diagram for data collection in at the TWD conference in 2013.

Analysis of Quantitative Survey Responses

We computed descriptive statistics, and used univariate analysis of variance (ANOVA) to test quantitative session survey questions for significant differences by participant gender (male vs. female); participant race/ethnicity (Black vs. other racial/ethnic minority group vs. white); and academic title (faculty vs. staff vs. administrator). A p value ≤ 0.05 indicated statistical significance; post hoc tests were run on items showing significant differences. Analyses were performed using SPSS (version 23.0, 2014; IBM).

Analysis of Qualitative (Open-Ended) Survey Responses

Two authors (A.K., A.F.) worked together to analyze qualitative responses from the participants. To fully capture the meaning of participants’ remarks, they leveraged a hybrid form of thematic analysis in which they first used quantitative questions as deductive codes and identified all text associated with a given question. To categorize the range of remarks for a given question, they then generated new inductive codes by open coding text associated with each question (Boyatzis, 1998; Braun and Clarke, 2006; Fereday and Muir-Cochrane, 2006). The two authors worked together to generate and assign codes. They then separately coded responses to five participants’ open-ended comments that were preselected to test interrater reliability (range 87–96%, Cohen’s kappa coefficient 0.91). The first author (A.K.) proceeded to code the remaining responses, and then the authors worked to combine codes into categories and larger themes. Responses to illustrate themes are provided verbatim. All qualitative analyses were performed using the NVivo qualitative software program (version 10, 2012; QSR International).

RESULTS

Session Survey

To illustrate scientific workforce trainers’ evaluation of Fair Play and their experiences playing, we present results from quantitative analysis followed by themes from qualitative analysis. Results from analyses of quantitative questions are presented together in cases in which qualitative themes overlapped for those questions.

Could Participants Discern the Objective of Fair Play?

To discern whether trainers could identify the intended purpose of Fair Play, we asked them to describe the game’s objective in open-ended comments. Qualitative analysis revealed that all participants’ responses fit into one major theme—Fair Play was viewed as a tool for learning about bias and its impact. Within this theme, many participants noted that the game particularly targeted bias experienced by underrepresented racial/ethnic minority graduate students. This theme was consistent in comments from male and female participants across racial/ethnic groups, as seen in these remarks:

“[The purpose of Fair Play is] to enable players to recognize and describe discrete scenarios in which an African-American student could encounter explicit or subtle bias in an academic setting.”—White male

“[The objective of Fair Play is] to teach and increase awareness of implicit bias and how [it] affects the lives of graduate students.”—Black female

“[The intended purpose of Fair Play is to help people] learn about the types of unconscious bias a graduate student might encounter.”—Hispanic female

“[The objective of Fair Play is] to teach people about implicit bias.”—White female

“[The objective of Fair Play is to] expose play[er]s to scenarios illustrating different situations with bias, [and] offer opportunities to identify bias.”—Native American male

Could Participants Take Jamal’s Perspective and Understand How He Felt during Bias Incidents?

To probe whether game play appeared to induce projective identity, which we theorized would help increase empathy for Jamal and help lower players’ implicit racial bias, we asked trainers whether they felt they could take Jamal’s perspective and understand how he felt during bias incidents. Quantitative survey responses to these questions showed that most trainers (85%, n = 40/47) agreed or strongly agreed that it was easy to take Jamal’s perspective and that they understood how Jamal must have felt during bias incidents (79%, n = 37/47; Table 3 and Figure 5). ANOVA results showed no statistically significant differences in these response items by participant race/ethnicity, gender, or title.

TABLE 3.

Quantitative survey responses for participants at the NIH TWD program directors’ meeting for fiscal year 2013 by race/ethnicity, gender, and agreement

Race	Gender	Easy to take Jamal’s perspective		Understood how Jamal must have felt in bias incidents		Playing was enjoyable		Occasionally felt uncomfortable while playing		Game accurately portrayed racial bias		Bias in game happens to African Americans in academia		Game is an effective way to teach about bias		Would play this game again in the future		Would recommend a colleague to play game		Could see a use for this game in professional setting or work
	Agree?	Yes	No	Yes	No	Yes	No	Yes	No	Yes	No	Yes	No	Yes	No	Yes	No	Yes	No	Yes	No
White	Female (n = 17)	15 (88%)	2 (12%)	10 (59%)	7 (41%)	8 (47%)	9 (53%)	7 (41%)	10 (59%)	14 (82%)	3 (18%)	15 (88%)	2 (12%)	13 (76%)	4 (24%)	11 (65%)	6 (35%)	14 (82%)	3 (18%)	15 (88%)	1 (6%)
	Male (n = 3)	3 (100%)	0 (0%)	2 (67%)	1 (33%)	1 (33%)	2 (67%)	1 (33%)	2 (67%)	3 (100%)	0 (0%)	3 (100%)	0 (0%)	1 (33%)	2 (67%)	1 (33%)	2 (67%)	2 (67%)	1 (33%)	2 (67%)	1 (33%)
African American	Female (n = 8)	6 (75%)	2 (25%)	7 (86%)	1 (13%)	6 (75%)	2 (25%)	1 (13%)	7 (86%)	7 (86%)	1 (13%)	7 (86%)	1 (13%)	7 (86%)	1 (13%)	7 (86%)	1 (13%)	7 (86%)	0 (0%)	8 (100%)	0 (0%)
	Male (n = 3)	3 (100%)	0 (0%)	3 (100%)	0 (0%)	2 (67%)	1 (33%)	0 (0%)	3 (100%)	2 (67%)	1 (33%)	3 (100%)	0 (0%)	3 (100%)	0 (0%)	3 (100%)	0 (0%)	3 (100%)	0 (0%)	3 (100%)	0 (0%)
Asian	Female (n = 3)	3 (100%)	0 (0%)	3 (100%)	0 (0%)	0 (0%)	2 (67%)	1 (33%)	2 (67%)	3 (100%)	0 (0%)	1 (33%)	2 (67%)	2 (67%)	1 (33%)	3 (100%)	0 (0%)	3 (100%)	0 (0%)	2 (67%)	1 (33%)
	Male (n = 2)	2 (100%)	0 (0%)	2 (100%)	0 (0%)	0 (0%)	2 (100%)	1 (50%)	1 (50%)	0 (0%)	2 (100%)	2 (100%)	0 (0%)	2 (100%)	0 (0%)	0 (0%)	2 (100%)	0 (0%)	2 (100%)	1 (50%)	1 (50%)
Hispanic/ Latino	Female (n = 4)	2 (50%)	2 (50%)	3 (75%)	1 (25%)	3 (75%)	1 (25%)	3 (75%)	1 (25%)	3 (75%)	1 (25%)	3 (75%)	1 (25%)	3 (75%)	1 (25%)	4 (100%)	0 (0%)	4 (100%)	0 (0%)	4 (100%)	0 (0%)
	Male (n = 2)	2 (100%)	0 (0%)	2 (100%)	0 (0%)	2 (100%)	0 (0%)	0 (0%)	2 (100%)	2 (100%)	0 (0%)	2 (100%)	0 (0%)	2 (100%)	0 (0%)	1 (50%)	1 (50%)	2 (100%)	0 (0%)	2 (100%)	0 (0%)
Native American/Alaska Native/American Indian	Female (n = 1)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	0 (0%)	1 (100%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)
	Male (n = 1)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	0 (0%)	1 (100%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	0 (0%)	1 (100%)
Unknown	Female (n = 1)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	0 (0%)	1 (100%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)	1 (100%)	0 (0%)
	Unknown (n = 2)	1 (50%)	1 (50%)	1 (50%)	1 (50%)	2 (100%)	0 (0%)	0 (0%)	2 (100%)	1 (50%)	1 (50%)	1 (50%)	1 (50%)	1 (50%)	1 (50%)	2 (100%)	0 (0%)	1 (50%)	1 (50%)	1 (50%)	1 (50%)
Total	Female (n = 34)	28 (82%)	6 (18%)	25 (74%)	9 (26%)	18 (53%)	15 (44%)	12 (35%)	22 (65%)	29 (85%)	5 (15%)	28 (82%)	6 (18%)	26 (76%)	8 (24%)	27 (79%)	7 (21%)	30 (88%)	3 (9%)	31 (91%)	2 (6%)
	Male (n = 11)	11 (100%)	0 (0%)	10 (91%)	1 (9%)	6 (54%)	5 (45%)	2 (18%)	9 (26%)	8 (73%)	3 (27%)	11 (100%)	0 (0%)	9 (82%)	2 (18%)	6 (55%)	5 (45%)	8 (73%)	3 (27%)	8 (73%)	3 (27%)

FIGURE 5.

Scientific workforce trainers’ (n = 47) responses to survey questions at TWD.

In open-ended comments, trainers were asked to explain what it felt like to be Jamal in an academic setting based on their experiences playing the game. Qualitative analysis revealed several themes. The first theme—being Jamal was a negative experience—was identified by trainers who agreed or strongly agreed that they could take Jamal’s perspective and/or understand how he felt (i.e., 85% of participants, n = 40/47). These participants used words such as “isolated,” “uncomfortable,” and “frustrating” to describe what it was like to be Jamal. Among the seven who indicated that being Jamal was not a negative experience, a few found the experience of learning and perspective taking as Jamal to be engaging, educational, and positive, while some made no clarifying comments. This theme was consistent across male and female participants and racial/ethnic groups and is illustrated by the following quotes from different participants:

“[Being Jamal was] isolating, overwhelming, [with a] lack of sense of belonging.”—Black male

“[Being Jamal] might be pretty uneasy and isolating.”—Hispanic female

“[Jamal is] not included by many and looked down on by others.”—Hispanic male

“[Being Jamal is …] quite uncomfortable.”—Black female

“[Jamal was] uncomfortable and possibly quite isolated.”—White female

“[Being Jamal] does not always feel comfortable.”—Asian female

“[Being Jamal is] challenging.”—Black male

“[It felt] confusing [to be Jamal].”—White female

“[Being Jamal was…] anxiety-inducing, and sometimes insulting.”—Black female

Within this theme, participants recognized the role that social interactions and the institutional environment played in perpetuating the bias and discrimination Jamal experienced, as seen in these remarks:

“Jamal is trying to find his way in academia and while people he encounters seem generally supportive, the environment is definitely biased against him even though the people he speaks to don’t realize this problem.”—White female

“[Jamal is] best described as on his own, [in an] insecure environment.”—Hispanic male

“Jamal probably feels somewhat isolated, due to some reactions and statements from personal encounters with community members, and some macroaggressions within the institutional environment.”—White female

Within this theme, trainers also acknowledged bias and discrimination, including others’ negative assumptions about Jamal, as sources of stress, and recognized how Jamal had to work harder to prove his competence, as seen in these comments:

“It is uncomfortable and frustrating [to be Jamal]. It is clear that assumptions are being made about Jamal.”—White female

“[Being Jamal was] isolating, discouraging, harder than necessary… he is plagued with doing and showing more than most students without sufficient support.”—Black female

“Jamal as a new student has the usual insecurities about fitting in to a new environment and particularly [for] establishing his credibility in a setting in which he is a minority. There is a bit of caution associated with each approach to a new person because of the anticipation of underlying assumptions about him and his character, whether his skills are at the expected level, whether he is in fact a credible, talented and promising student.”—White male

“[Jamal is] a fish out of water. With each person you approach, there is an extra stress because you are always wondering what people are really thinking about you before you even say hello. As if grad school is not stressful enough!” Hispanic female

Further analyses of the quantitative and qualitative data indicate that, among those who responded that they could both take Jamal’s perspective and understand how he felt, only female participants wrote about being Jamal in the first person. Specifically, 24% of the female participants who responded positively to both of those questions (n = 6/25; Table 3) used the first person. Comments from these female participants differed in subtle ways from the third-person remarks above in that they were more elaborative and specific. For example, they described feeling “invisible,” “alone,” “friendless,” and “left out,” which went beyond participants’ third-person remarks that Jamal was isolated. They also described feeling “lost,” “alienated,” “awkward,” and like they had to be “on guard,” but at times also felt “encouraged,” which elaborated participants’ third-person remarks about Jamal’s confusion, discomfort, and experience of being misunderstood. The following quotes illustrate this theme:

“[I felt] frustration, isolation.”—Black female

“I felt a bit lost […] I felt like I was a bit invisible.”—White female

“[I felt] a little alienated but sometimes encouraged. Often misunderstood.”—Asian female

“[I felt] awkward, on guard for incidents of bias.”—White female

“[I felt] quite alone; friendless and left out.”—Asian female

“[I felt] very uncomfortable.”—White female

A minority of participants indicated that they were unable to easily take Jamal’s perspective (female [F] = 6/34, 18% vs. male [M] = 0/11, 0%) and/or were unable to understand how Jamal felt during bias incidents (F = 9/34, 27% vs. M = 1/11, 9%). At the same time and similar to their counterparts who indicated they could take Jamal’s perspective and understand how Jamal felt, qualitative analysis of open-ended comments showed that all participants in this group were able to articulate what it felt like to be Jamal. Notably, a larger proportion of female than male participants were in this group. The following comments show how these participants were able to express how it felt to be Jamal despite indicating on quantitative items that they could not easily do so:

“[Being Jamal felt] frustrating!”—White female

“I felt his experience was fairly isolating.”—Black female

“[As Jamal I felt] intimidated, lost, unwelcome.”—White male

“The experience [of being Jamal] can be a bit discouraging.”—Hispanic female

“[It was] sometimes uncomfortable [to be Jamal].”—White female

“[I felt] frustration, isolation.”—Black female

“[Being Jamal was] confusing—concerning mixed messages about feeling welcome.”—White female

Within this theme only one participant, a Black female, explained why she found it difficult to take Jamal’s perspective:

“I’ve experienced or have friends who have experienced very similar scenarios of racial bias so I thought that they were accurate. I think it was hard to take Jamal’s perspective because I didn’t have enough information to feel like I really ‘knew’ him. Maybe a section or menu selection that would tell you more about his background would help someone be more sympathetic and connected to his experience—especially those who have never faced these scenarios before.”—Black female

In summary, all trainers, even those who indicated in quantitative responses that they could not easily take Jamal’s perspective or understand how he felt in bias incidents, were able to explicate how it felt to be Jamal in qualitative comments. Only female participants used the first person to describe their experiences as Jamal—suggesting a strong identification with the avatar. In these cases, perhaps they were truly feeling empathy and were able to describe feelings that they have felt themselves. Somewhat paradoxically, a greater proportion of female than male participants also indicated they did not feel that they could take Jamal’s perspective or understand how he felt, despite all participants clearly articulating how it felt to be Jamal in open-ended comments. This finding is difficult to interpret, but may be due to the difference between being able to take someone’s perspective as compared with having feelings in that situation. Both are needed for projective identity. There were no distinctive thematic patterns by participant race/ethnicity.

Did Trainers Think the Game Accurately Portrayed Racial Bias, Biases in the Game Happen to Blacks in Academic Settings, and Did They Feel Comfortable during Game Play?

Accuracy of the game’s content is critical for effectively boosting players’ bias literacy, and survey results showed that a majority of trainers agreed or strongly agreed the game accurately portrayed racial bias (81%, n = 38/47) and that biases in the game happen to Blacks in academic settings (85%, n = 40/47). Playing was not always comfortable, however, as 30% of trainers (n = 14/47) agreed or strongly agreed that they occasionally felt uncomfortable during game play. For the latter item, ANOVA results showed that significantly more female (35%, n = 12/34) than male (11%, n = 2/19) and more white (38%, n = 8/21) than Black (9%, n = 1/11) participants felt uncomfortable playing (p values < 0.05). Exploration of descriptive statistics showed that, while only two male participants (18%, n = 2/11, one Asian and one white) reported feeling uncomfortable, 30% of female participants (n = 10/34) reported discomfort, and these female participants were mostly white and Hispanic. Specifically, 41% of white (n = 7/17) and 75% of Hispanic female participants (n = 3/4) reported discomfort compared with only one Asian (33%, n = 1/3) and one Black female participant (13%, n = 1/8) reporting discomfort (Table 3).

Qualitative analysis of open-ended comments revealed that these three items were interrelated for scientific workforce trainers. We found that players who indicated they had a high level of knowledge about bias (either from lived experience or from scholarship) appeared to feel most comfortable playing the game and used their knowledge to rate the content of the game as accurate. Response patterns differed by player gender and race/ethnicity, however, as the three emergent themes revealed.

The first theme—using knowledge from personal experience to validate the accuracy of the game—occurred for only male and female Hispanic participants and Black female participants who indicated they felt comfortable playing the game, as seen in these quotes:

“I don’t think I felt uncomfortable because I’m an African American female and these experiences are not new to me. Making others uncomfortable is a good outcome of this game…that will get them thinking.”—Black female

“[To assess the accuracy of the game] personal experience and observation were factors.”—Hispanic male

“I am AA and I experienced most of it while in graduate school.”—Black female

“I have observed and experienced some of the bias demonstrated in this game. It really helped me to recognize and identify the different types.”—Hispanic female

The second theme—use of external information, from either scholarship or observations of bias, to evaluate the accuracy of the game—occurred for only male and female white and Asian participants who indicated they felt comfortable playing the game. Within this theme, these participants also acknowledged how not being Black limited their ability to assess the extent to which biases in the game happen to Blacks in academic settings. The following quotes illustrate this theme:

“The biases in this game were fairly true to form. I found it interesting that Morgan [a white male character] seemed so pontificating after he had taken a course on racial bias. This hits pretty close to home, as I have made some of the same statements in my years. This game uses good research and citations. You’ve done your homework.”—White female

“[T]he examples of bias are good.”—Asian male

“I was familiar with these situations as I have observed them in real time settings.”—White female

“Since I am not one [i.e., not African American] my experience may be different.”—Asian female

The third theme—discomfort, or worry and concern about the game content—surfaced for white and Hispanic female participants and an Asian male participant who indicated they felt uncomfortable playing the game. The following quotes illustrate this theme:

“I found the game easy to play but the content was disturbing, not enjoyable, although I did feel like laughing at some of the comments.”—White female

“[The game was] unsettling, […] because of [the] topic.”—White female

“I am concerned about who the user or intended audience for the game is. It might not be appropriate for students but might help faculty.”—Hispanic female

“This seems like a possible means of learning about biases that URM students might face, and help them overcome biases. But I wonder if focusing on their challenges in grad school would reinforce a deficit model towards diversity.”—Asian male

Interestingly, others who indicated that they felt uncomfortable playing the game thought the game was helpful for teaching and learning about bias, as seen in this quote:

“[The game] helps [players] understand comments and actions that might be considered biased.”—White male

Only one Black participant, a woman, reported feeling uncomfortable during game play, but noted that this was a consequence of the game’s accuracy. She did not appear concerned about the game’s content nor did the content diminish her interest, as she explains,

“I felt uncomfortable at times because I know how it feels to be the target of racial bias, so that’s a testament again to the accuracy of the game scenarios. I only played two chapters, but I was intrigued enough that I would have played more if it was available. I would recommend to others.”—Black female

In summary, these data suggest that a lack of familiarity with bias (either scholarly or from lived experience) may lead players to feel discomfort when they play the game and encounter bias as Jamal. Among players in this sample, white and Hispanic female participants appeared to be the most likely to feel uncomfortable playing the game, potentially because they had different lived experiences than what they encountered as Jamal or less scholarly knowledge about bias. Trainers who reported feeling uncomfortable appeared to be more willing to voice concerns about the extent to which the game could have a negative impact on some players.

Did Trainers Think That the Game Was an Effective Way to Teach about Bias, Would They Recommend the Game, Could They See a Use for It in Their Work, Did They Enjoy Playing, and Would They Play Again?

This final set of questions probed the extent to which trainers considered Fair Play useful for teaching about bias and would consider using it themselves. These questions helped us gauge the potential feasibility of disseminating the game. A majority of participants agreed or strongly agreed the game was an effective way to teach about bias (79%, n = 37/47), that they would recommend this game to their colleagues (86%, n = 40), that they could see a use for Fair Play in their professional settings or work (83%, n = 39), and that they would play the game again in the future (75%, n = 35/47). By comparison, only about half of participants (55%, n = 26/47) agreed or strongly agreed that they enjoyed playing the game. ANOVA showed no statistically significant differences by participant gender, race, or title for these items.

Qualitative analysis of open-ended comments revealed that these questions were interrelated for participants and yielded two themes. Comments suggest that participants’ ability to understand how to play the game and move Jamal around—game mechanics—was linked to the extent to which they saw the game as a good teaching tool, enjoyed playing, and could see a use for it in their professional settings or work. Specifically, this theme revealed that participants who struggled more with game mechanics or with discerning what Jamal was supposed to do appeared to be less likely to see the game as an effective way to teach about bias, enjoyed playing less, and did not see a use for it in their professional settings or work (11%, n = 5/47). The following quotes illustrate this theme by showing participants’ struggles:

“The content was fine, but the mechanics of the game were cumbersome.”—White male

“It was a little difficult to maneuver which took away from the goals of the game.”—Black female

“I found the game tough to navigate and it took a long time to make some points that could have been made more quickly.”—White male

“It was not always clear where things were going; confusing choices?”—Asian female

“I wondered what the ‘game’ was. Reaching my goals in less time? Correctly identifying bias?”—Native American male

The second theme—seeing the game as useful and planning to use it to address bias—included responses from all other participants (89%, n = 42/47; i.e., all those who did not report struggling with game mechanics). Importantly, each of these participants articulated at least one way that he or she could use the game or planned to use the game. The following comments illustrate participants’ perspectives on the usefulness of the game:

“Many do not understand implicit bias and this might help explain and break the ice for a conversation.”—Native American female

“I could definitely send this out to my graduate students and faculty in order to help them understand this problem on campus.”—White female

“I would use this with my students who are planning to attend graduate school.”—Black female

“[I plan to use it for] faculty development.”—Black male

“If given to faculty, it could make them aware of their own implicit biases…and hopefully help them to alleviate them, or at least change their behavior.”—Black female

“Addressing bias, use in orientation for new students and faculty.”—White female

Within this theme, participants remarked that the platform of a game could be helpful for addressing other types of bias, as these quotes illustrate:

“Could be used for sexual harassment training as well!”—White female

“Please keep working with this game and developing ways to customize it to many different situations.”—White female

Others noted that the game made them reflect on their own experiences at their institutions and the messages that underrepresented racial and ethnic minorities might be receiving, as this quote illustrates:

“We have a wall EXACTLY like the one that Jamal encounters [Figure 2]. This is the first thing I’ll address when I return home.”—White female

Within this theme, participants also identified ways they would like to see the game improved. For example, some remarked that the game would be most useful in a facilitated workshop, while others said they would have liked to choose their own avatars. Having a choice would help them relate more to the character and learn to how to address racial bias in ways that they could apply to their real-world identities:

“[The game would be most useful in] a workshop on implicit bias.”—Black male

“I applaud the effort to find a way to tackle this sensitive topic. I think it’s hard to fully capture a complex topic such as racial bias in a game, but I do think it could be effective if paired with facilitated conversations after playing.”—Black female

“I might have learned more from being on the other side, [i.e., as one of the characters who interacted with Jamal] and given multiple possible responses which could reveal my biases. Would I learn more by empathizing with Jamal or with the other characters?”—Native American male

“It would be interesting to choose a character at the very beginning and to have each character have different experiences. Of course, this makes everything far more complicated but it would allow for multiple plays with different outcomes.”—White female

“Having an avatar that was adaptable to my race and gender could help me personally. Selecting other avatars to learn how the stereotype/bias affects them.”—White female

In summary, most trainers viewed the game as useful for teaching about bias and articulated ways they could use or planned to use the game in their professional work; some of these participants shared ways that the game could be enhanced, such as incorporating it into a workshop or providing players the opportunity to select an avatar with specific characteristics. Interestingly, it was not the game content but rather challenges related to the mechanics of the game, such as maneuvering Jamal and understanding what he was supposed to do, that made the remaining minority of participants indicate that they did not enjoy playing, did not see the game as a good teaching tool, and/or did not see a use for the game in their professional settings or work. This feedback provided important information for ways to improve and expand the scope of the game.

DISCUSSION

In this mixed-methods study, a group of scientific workforce trainers—which included science faculty—evaluated Fair Play, an avatar-based role-playing video game designed to address implicit and subtle racial bias in science through perspective taking and the promotion of bias literacy. The premise of Fair Play is that role-playing will allow players to take Jamal’s perspective through the cognitive process of projective identity—defined as a cognitive merging or shared identity of a player with an avatar that arises as a consequence of depending on that character to successfully navigate a game domain (Gee, 2007). Perspective taking has been shown in several experimental paradigms to reduce racial bias and promote positive intergroup behaviors by cultivating empathy for members of a stereotyped group (Batson et al., 1997; Galinsky and Moskowitz, 2000; Galinsky and Ku, 2004; Galinsky et al., 2005; Bilewicz, 2009; Shih et al., 2009; Todd et al., 2011). In the context of Fair Play, if role-playing can induce projective identity and allow for perspective taking and the experience of empathy for Jamal, this could reduce players’ implicit racial bias and biased behavior directed toward Blacks.

In this study, quantitative results showed that the majority of participants reported they could take Jamal’s perspective and understand how he felt in bias incidents. Importantly, qualitative analysis of open-ended comments about how participants felt as Jamal revealed that even participants who indicated that they could not easily take Jamal’s perspective and/or did not understand how he felt on quantitative scales were able to clearly articulate how it felt to be Jamal in qualitative comments. In qualitative comments, participants appeared to not only take Jamal’s perspective but to feel empathy for him, as they openly expressed feeling frustrated, isolated, anxious, and burdened by being judged by others and having to continuously prove their competence. They also clearly articulated the psychological impact of encountering bias and discrimination as Jamal and described the way bias disadvantaged them in completing the tasks necessary to succeed in graduate school.

Taken together, these results provide compelling evidence that trainers may have experienced projective identity as Jamal, which requires both taking the perspective of and feeling with a character. This finding is promising regarding the game’s potential effectiveness for reducing racial bias in science faculty. An earlier version of the game (Table 1), tested on science graduate students at the University of Wisconsin–Madison, showed that players had lower implicit racial bias scores, as measured on the Implicit Association Test, when they were able to take Jamal’s perspective and empathize with him (Gutierrez, Kaatz, et al., 2014). Future studies should test the extent to which perspective taking leads to empathy and lowers implicit racial bias in science faculty.

Related to this finding, most trainers indicated that they learned a lot about bias from playing the game. Other work on educational games shows that projective identity can allow players to experience embodied cognition through their avatars—a state of active learning that arises from physical and psychological experiences (Gee, 2007). This deep learning process is required to develop domain literacy. Trainers’ reports that they learned about bias through game play may indicate that they had an increase in bias literacy, which would also be expected to reduce implicit bias (Devine et al., 2012; Carnes et al., 2015). Another important direction for future research will be to directly examine the extent to which Fair Play promotes bias literacy in science faculty.

Interestingly, results showed that only female participants described what it was like to be Jamal in the first person. Paradoxically, we also found that proportionally more female than male participants indicated in quantitative responses that they could not easily take Jamal’s perspective and/or could not understand how he felt in bias incidents. Female participants in both of these groups were representative across racial/ethnic subgroups, which suggests that factors beyond race/ethnicity may play an important role in allowing female players to develop empathy for Jamal. Scholarship on avatar self-relevance suggests that players may be better able to form an emotional connection with an avatar when there is gender and race concordance (Ratan and Dawson, 2016). Other research suggests very little “gender swapping” actually occurs when playing video games. In other words, players, when given the choice of an avatar’s gender, choose one different from themselves only 16% of the time on average. Men, however, are more likely to do so than women (17.4 vs. 8.2%; Huh and Williams, 2010). In our study, some players who indicated that it was hard to take Jamal’s perspective and/or understand how he felt suggested that the game would be more beneficial for them if they were able to select the gender, race/ethnicity, and/or role of the avatar character, as this would allow them to better relate to their avatars and acquire skills for identifying and addressing bias in roles more similar to their own. A future direction for Fair Play could be to integrate an option for players to select characteristics of their avatars and to study the extent to which avatar self-relevance is a mechanism through which players form an emotional/empathic connection to their avatars.

Most trainers rated the game as accurate and thought that biases in the game happen to Blacks in academic settings. For study participants, questions about the accuracy of the game appeared to be related to how comfortable they felt while playing and the extent to which they had personally experienced bias or had scholarly knowledge about bias. Statistical analyses showed that significantly more female than male and more white than Black trainers felt uncomfortable playing the game. These findings may reflect differences in both individual players’ lived experiences and in how they react to bias (Batson et al., 1997; Galinsky and Moskowitz, 2000; National Academy of Sciences, National Academy of Engineering, Institute of Medicine, 2005; Todd et al., 2011). Reactions to bias can be influenced by factors such as domain self-efficacy, self-esteem, and social support (Batson et al., 1997; Galinsky and Moskowitz, 2000; Todd et al., 2011), which we did not measure. Qualitative analyses of open-ended comments by participant gender and race/ethnicity helped to explain these results. We found that the group of trainers who felt uncomfortable playing the game included mostly white and Hispanic female participants—in comments, none of these participants talked about the accuracy of the game, indicated they had experienced bias, or indicated they had scholarly knowledge about bias. These trainers also appeared to be more likely to raise caution about the content of the game. By comparison, among trainers who indicated that they felt comfortable playing Fair Play—a group in which Black participants were overrepresented—analysis of qualitative responses showed that Black and Hispanic participants appeared to use their lived experiences of bias to rate the game as accurate, while white and Asian participants appeared to use their scholarly knowledge of bias to rate the game as accurate. These findings help to provide support that Jamal’s experience in the game aligns with underrepresented racial/ethnic minorities’ real, lived experiences of bias in academic settings and with scholarly documentation of bias. They also suggest that playing the game may be quite an uncomfortable experience for players whose life experiences and scholarly knowledge do not encompass the type of bias Jamal experiences in the game.

A majority of trainers and faculty indicated that they planned to use the game in their work or at their home institutions and identified a range of uses for the game, including professional development and training for faculty, students, and staff. Even players who felt discomfort indicated that they planned to use the game in the future. Qualitative finings revealed that those who did not indicate they planned to use the game struggled with game mechanics. As mentioned earlier, participants also suggested that having the opportunity to select the gender, race, or role of their avatars could make the game more high-yield by providing the opportunity to learn about bias and its impact from multiple perspectives. Again, this aligns with research on avatar self-relevance, which suggests avatar personalization can enhance identification with a game character (Ratan and Dawson, 2016). Future work should target improving the game to make it more accessible for nongamers and explore the utility of an adaptable avatar.

Although Fair Play was viewed positively by trainers as a means to facilitate discussion of the sensitive topic of racial bias in academic science, we do not know whether playing Fair Play for a short time can have an enduring and meaningful impact on a player’s attitudes and behaviors in the context of racial bias. If playing the game can make faculty and staff in science more bias literate, however, they will be better able to identify and label racial bias when they observe it occurring in academic environments. Sevo and Chubin (2008) posit that, just as literacy is fundamental to new knowledge creation in other realms, in the area of intergroup bias, a common vocabulary should enable individuals to converse and problem solve. In Nonaka’s (1994) model of organizational change, such literacy facilitates discourse, which is fundamental to the process of creating new knowledge and transferring it among individuals within an organization to ultimately achieve broader organizational cultural change. We have previously demonstrated that improving gender bias literacy among faculty in academic science, engineering, and medicine departments with a workshop intervention increased measures of intentional behavioral change and improved department climate (Carnes et al., 2015). Taken together, our results suggest that Fair Play could be a useful way to promote racial bias literacy as an intervention to promote a more inclusive culture in academic science. The game is now part of a workshop that includes research about bias in the academy, playing the game, reflecting on the experiences of Jamal, and strategies to reduce bias, all of which are essential aspects of bias literacy.

Because the vast majority of individuals in academic science are white (Nelson, 2007), we were mindful that white players would be the primary target group for Fair Play. At the same time, we did not want the game to be offensive or patronizing to players from racial/ethnic underrepresented groups. The attendees at the TWD meeting are a self-selected group, most of whom are interested in greater scientific workforce diversity, and they are more racially/ethnically diverse than the overall field of academic science. This may have made study participants more willing to view the game in a positive light, particularly as a teaching tool about bias. The finding that participants who appeared to be the least familiar with bias were those who felt most uncomfortable playing the game may suggest how faculty would react to the game. This is an important consideration before the game is widely disseminated. Although having highly diverse groups of participants in the study may be a limitation in generalizing the results of our research, it provided an opportunity to explore the impressions and experiences of individuals who had been targets of the type of bias experienced by Jamal. We were gratified to find that both white players and players from underrepresented racial/ethnic minority groups found the game useful and viewed Jamal’s experiences within the game as authentic.

In summary, we have presented the rationale for using game-based learning as a means to introduce science faculty and staff to concepts of implicit racial bias and described the development of the interactive video game Fair Play. We conclude that Fair Play may be a useful way to introduce and promote discourse around the sensitive topic of racial bias, given the study’s results from scientific workforce trainers who are interested in scientific workforce diversity and who would be highly qualified to comment on its potential utility. Fair Play also appeared to promote bias literacy and to foster perspective taking, both of which are effective means to help change cultural norms in ways that will afford Blacks equal opportunities to contribute to the scientific enterprise.

Accessing Materials

A complete list of bias concepts, definitions, in-game examples, and references included in Fair Play is given in Table 2. For more information, or to play the current version of the game, visit www.fairplaygame.org. The survey instrument used to evaluate the game can be found in Appendix A in the Supplemental Material.