Skip to main content

Some NLM-NCBI services and products are experiencing heavy traffic, which may affect performance and availability. We apologize for the inconvenience and appreciate your patience. For assistance, please contact our Help Desk at info@ncbi.nlm.nih.gov.

NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jan 1.
Published in final edited form as: J Women Minor Sci Eng. 2021;27(2):87–106. doi: 10.1615/JWomenMinorScienEng.2021032729

Improving Department Climate Through Bias Literacy: One College’s Experience

Jennifer Sheridan 1, Eve Fine 1, Manuela Romero 1, Carmen Juniper Neimeko 1, Molly Carnes 1, Christine Bell 1, You-Geon Lee 1
PMCID: PMC8159150  NIHMSID: NIHMS1658475  PMID: 34054279

Abstract

Many institutions of higher education are investing in “implicit bias training” as a mechanism to improve diversity and inclusion on their campuses. In this study, we describe an effort to implement this training in the form of a 3-hour workshop delivered to faculty members in the College of Engineering at the University of Wisconsin-Madison. Evaluation form data collected immediately post-workshop, and in-person interviews and survey data collected 6–12 months post-workshop, were used to measure the effectiveness of the intervention. These data show that faculty awareness of implicit bias in their workplace environments increased significantly, although individual motivation and self-efficacy to act without bias, and self-reported bias-reduction actions, did not increase. At the same time, we found evidence of improved department climates and bias-reduction actions at the department level, which increase our confidence that the workshops were having a positive impact. Importantly, women and faculty of color in the College did not report increases in negative behavior after the workshop, and reported that their departments were engaging in explicit discussions of potential biases in departmental processes more often. These findings support the continued implementation of the “Breaking the Bias Habit®” workshops along with measurement of their success.

Keywords: Implicit bias, faculty, STEM, intervention studies

1. Introduction

“Implicit bias training” has become a recommended solution for any educational unit wishing to address diversity and inclusion issues within their institution. Most institutions with National Science Foundation ADVANCE programs incorporate implicit bias concepts into training for faculty search committees and often for tenure/promotion committees as well (Bilimoria and Liang, 2012; Stewart et al., 2007; Stewart and Valian, 2018). The American College of Physicians has explicitly called for implicit bias training on a regular and recurring basis by all organizations that employ physicians (Butkus et al., 2018). More examples of the spread of implicit bias training in academic settings can be found by conducting a simple internet search for “university implicit bias training.” Some universities offer implicit bias trainings (e.g., Vanderbilt University and the University of Oregon), while others provide resources for self-education (e.g. Northwestern University and the University of Massachusetts Amherst)(Google search, October 2020).

The goals of these workshops, training experiences, and resources typically include reducing bias at evaluation points (e.g., hiring more diverse faculty/staff, admitting more diverse students, or creating equitable promotion processes); retaining diverse faculty, staff and students; and improving department and campus climate. It is difficult to know whether these trainings are meeting these goals, as very few trainings evaluate their impact on these outcomes. A few studies that look at outcomes of implicit bias training in academic settings show that it can be effective at changing behaviors based on implicit bias (Forscher et al., 2017) and, at least in the gender context, can lead to improving department climate (Carnes et al., 2015) and hiring more diverse faculty (Sheridan et al., 2010; Devine et al., 2017). However, studies in a corporate context show that some forms of “diversity training” not only do not work, but increase resistance to the concepts they impart (Dobbin and Kalev, 2016).

1.1. Individual and Organizational Change

In our efforts to realize positive outcomes of implicit bias education in academia such as diversity in hiring and promotion, retention of a diverse faculty, and a welcoming and inclusive climate, we recognized that change must come at an individual level as well as an organizational level to improve the culture of a department. At the individual level, we structured our educational intervention with the idea that implicit bias functions as a cognitive and behavioral habit. Prejudiced actions can occur through unconscious (implicit) processes that may contradict one’s conscious (explicit) beliefs (Devine, 1989). Individuals are frequently unaware of these habitual, implicit processes, or “habits of mind.” To break these habits of mind, a number of stages must be passed through (Bandura, 1977; Carnes et al., 2005; Prochaska and DiClemente, 1983; Prochaska and Velicer, 1997). In the first stage (“precontemplation” or “unconscious incompetence”), individuals are not aware of any problems related to the lack of diversity in our educational institutions, do not realize that their colleagues from underrepresented groups may be experiencing a more hostile working environment, or do not imagine that they themselves might be behaving in biased ways. As awareness is raised about these issues, individuals move to a stage of “contemplation,” where they may begin to realize they might need to change their behavior, and they might begin to notice inequities in the environment that they did not notice before. At this stage, they may be looking for motivation to engage in behavioral change. They might exhibit bias behaviors at this stage, but they are becoming more conscious of those actions (“conscious incompetence”). Next, individuals prepare to make changes in behavior by increasing their self-efficacy for recognizing and interrupting biased thoughts. They attend workshops, read blogs or literature about the experiences of underrepresented groups, and talk to others to find out what actions they had successfully used to promote diversity and inclusive behavior in their work environments. When ready, they finally take action, becoming “consciously competent” to reduce implicit bias in individual actions and within the work environment. Over time, these new actions are maintained until they become habitual (“unconscious competence”, Carnes et al., 2012).

While individuals may work to change their personal habits of mind that lead to biased actions, institutional change happens when the entire departmental unit is working in concert to create new norms that support non-biased behavior. Without the ability to challenge biased actions in self and others or to examine departmental practices and policies that may be contributing to bias against underrepresented individuals, it is unlikely that individual change alone can change an organization to make it more inclusive and welcoming to all (Forscher, 2016; Paluck and Shepherd, 2012; Centola, 2013).

1.2. Breaking the Bias Habit®

Beginning in 2009, we began creating workshops that incorporated these elements of both individual behavioral change and organizational change. Using the “habit of mind” metaphor developed by Devine (1989), we created a workshop designed to help faculty participants break their “bias habits” and therefore move through the stages of change by:

  • Increasing awareness of one’s own bias, through increasing bias literacy (Sevo and Chubin, 2008; Howell, 1982; Nonaka, 1994) and exploring one’s own biases via an Implicit Association Test (IAT, Greenwald et al., 1998);

  • Motivating participants to want to work on overcoming the influence of implicit bias “habits of mind” by informing them of the experiences of underrepresented persons in academia and generating multiple benefits of improving diversity and inclusion in their units, thereby increasing positive outcomes expectations of engaging in bias-reduction activities;

  • Providing evidence-based, concrete strategies that individuals can practice to break their own implicit bias habits, as well as suggesting actions that a department or working group can take to reduce bias in the unit, thereby increasing self-efficacy to act without bias; and

  • Practicing some of these strategies within the workshop as well as naming specific actions one can take—both at the individual and departmental level—to reduce implicit bias.

To ensure that the strategies and actions discussed in the workshop began to change the department’s culture around these issues, the workshop was implemented at a department level; that is, the workshop was only held within a departmental or other academic unit, with all participants belonging to the same unit, rather than a campus-wide open workshop where the participants could come from any of hundreds of departments.

From 2009 to approximately 2015, the workshop we developed focused exclusively on implicit gender bias (Carnes et al., 2012). Using the framework above we conducted an experimental study, randomly assigning some departments/units to receive the workshop and others to be controls. We found that participants in these workshops did increase their awareness of their own personal bias, their motivation to act without bias, and their self-efficacy. They also self-reported acting with less bias, but importantly only if 25% or more of the department attended the workshop (Carnes et al., 2015). This result indicated that having a critical mass of faculty in a department who also are working on their implicit bias habits is necessary to creating a departmental culture that allows for such action.

In 2016, we began to expand the content of these workshops. We wanted to include information about implicit bias beyond gender bias, especially to include racial/ethnic bias. We revised the workshop in the following ways:

  • Strong focus on both gender and racial/ethnic implicit bias, with some mention of biases around disability and LGBT issues;

  • Expand the time from 2.5 to three hours to allow for more discussion;

  • Rearrange the content so that strategies to combat implicit bias follow directly upon the implicit bias topic covered;

  • Include information on new topics such as microaggressions, which are brief or subtle comments, behaviors, or environmental cues that communicate hostile, derogatory, or unwelcoming messages towards members of groups underrepresented in their environments (Sue, 2010; Fine et al., 2018).

The new workshop, entitled “Breaking the Bias Habit®: A Workshop to Promote a Diverse, Welcoming, and Inclusive Campus,” was delivered to all eight academic departments in the University of Wisconsin-Madison (UW-Madison) College of Engineering (CoE) between January and August, 2017. Workshop materials, including the agenda, concepts covered, case study, and other workshop handouts, are included in the Supplementary Materials. Endorsed at the highest levels by the Dean, Associate Dean, and Department Chairs, attendance was very high, with over 85% of all CoE faculty participating in the workshops. Because the entire CoE was committed to participating in these workshops, we were not able to conduct a randomized, controlled experiment to study the effectiveness of the new intervention. However, we took a number of steps to measure both individual change and departmental change with regards to changing implicit bias habits.

2. Method

We used a mixed-methods approach to understand the impacts of our implicit bias workshop in the College of Engineering. Evaluation form data from 114 faculty and staff who attended the workshops were collected immediately after each workshop in 2017. Interviews with 18 faculty and staff were conducted from late 2017 through early 2018. Finally, survey data were collected 7–12 months post-workshop (early 2018) from 154 faculty and staff respondents. We combined data from all of these sources to inform our understanding of the impact of our intervention. Data collection for the in-depth interviews, the surveys, and the workshop evaluation data was approved by the UW-Madison Institutional Review Board (#2016–1510; #2017–0001).

2.1. Authors’ Positionalities

The research team was comprised of the workshop presenters as well as other researchers who gathered and analyzed outcome data. Members of the research team were cisgender women and one man. Most of the team was white, with African American, Hispanic, and Asian identities also represented. Most of the authors were academic staff, both within and outside the CoE, but some were faculty and some were graduate students at the time of this study. Team members had diverse disciplinary backgrounds, with training in biological sciences, social sciences, and humanities. Most of the team members were born in the U.S., but some were not. We were intentional about forming a diverse team, especially for the face-to-face interviews, so that participants who were underrepresented might share their perspectives more freely with someone who shares some of their identities. Several strategies were employed to protect all research participants, particularly those participants who identify as being underrepresented in their departments. The post-workshop surveys and evaluation forms were anonymous, so any information collected about participant identities was both voluntary and did not identify individuals. Information about interview participants’ gender or race/ethnicity was not combined with rank or department when quoted. These strategies were important, as some departments had (for example) only one assistant professor of color.

2.2. Evaluation Form Data

After every workshop, participants were asked to fill out a two-page evaluation form, measuring how much they valued of specific elements of the workshop and knowledge gains in the various topic areas. No demographic or other identifying information was collected from these forms; they were anonymous for each participant, although the department was known because they were administered at the end of a department’s workshop. Overall, we received 148 forms from the 226 workshop participants in our eight CoE departments, for a 65.5% response rate. The evaluation form is included in the Supplementary Materials. Evaluation form data only reflect post-workshop opinions; we did not conduct a pre-workshop survey.

2.3. In-Depth Interviews

We developed an interview protocol to explore in depth with eighteen faculty and staff participants the climate in the department, and changes (positive or negative) that may have resulted from the workshop (see Supplementary Materials). The interview questions asked participants to: (1) describe their department’s climate; (2) share any personal changes from their participation in the workshop (for those who attended); (3) describe any noticeable departmental changes since the workshop; and (4) make suggestions for workshop improvements. When selecting Engineering faculty/staff to participate, care was taken to select: 2–3 persons from each of the eight departments; department staff members as well as faculty; all ranks of faculty; both male and female faculty; several faculty/staff of color; some persons who attended the workshop and some who did not. For purposes of selecting CoE members for interviews, faculty/staff of color are defined as listing their racial/ethnic category in official university data as American Indian/Alaska Native, Asian, Black/African American, Hispanic/Latino, Native Hawaiian/Other Pacific Islander, or 2 or more races, regardless of country of birth or immigration status. Approximately 17.5% of faculty and staff in the College of Engineering identify as faculty/staff of color using this definition. We attempted to select both under-represented and well-represented non-white interviewees. Within these categories, Dr. Jennifer Sheridan, PI of this project, a research staff member in the College of Engineering, and one of the workshop presenters, selected and contacted individuals and invited them to participate in an interview. We selected Dr. Sheridan to invite the interview participants because she was known personally to most respondents due to her role as a workshop presenter, but Dr. Sheridan did not perform any interviews herself. We reasoned that respondents would be more truthful about their workshop experience if the interviewer was not closely associated with the workshop development and delivery. We invited 37 faculty/staff to participate in an interview, and 19 agreed. One interviewee declined to sign the consent form, and therefore those data were not included in this study. Two researchers (specifically, authors Christine Bell and Casey Stockstill) conducted the one-on-one interviews which lasted from 30–60 minutes. The interviewers were both female, one white and one African American. Both interviewers were unaffiliated with the College of Engineering, an intentional decision to encourage the respondents to explain in more detail their answers (since the interviewers don’t “know” the Engineering context). The interview was audio-recorded, interviewee assigned a pseudonym, and the recording transcribed. NVivo software was used to facilitate descriptive coding in the transcripts and organize the codes into the major themes (Braun and Clarke, 2012). The results of the quantitative survey did not influence the coding of the data, as the interview coding and the survey data analysis occurred concurrently. The two interviewers/researchers began by independently developing descriptive codes for the interviews, then each researcher grouped the descriptive coding conducted by the other researcher in establishing a codebook. Examples of the 65 descriptive codes we created included “climate in department meetings” or “awareness of bias incidents.” Then, the researchers met to solidify the descriptive codebook and identify themes that respond to the research questions. Ultimately, the data coalesced into 17 themes. Examples of themes include “department climates were described as mediocre or business-like” or “less interruptions during department meetings.” Next, the researchers applied the descriptive coding and identification of themes to every transcript. All transcripts were coded by both researchers and discrepancies in descriptive coding and identification of themes were resolved by discussion. The researchers prepared memos in the form of a synopsis of each interviewee’s experience with their department’s climate and thematic highlights from each interview at the end of the coding process. Preliminary findings from the subset of themes that directly related to the workshop effects were shared and discussed with the research team for confirmation.

2.4. Surveys

To further explore whether the workshop initiated change in climate in the CoE, we created a short survey, available in the Supplementary Materials. The survey, separate and different from the evaluation form described in Section 2.2, was administered in person, via a visit to each of the 8 department meetings in spring semester, 2018. At the faculty meetings, a study team member reminded the faculty/staff attending the meeting about the department’s participation in the workshop and asked them to please fill out a short survey. The survey instrument had a selection “Choose to not respond” for every item on the survey, so that meeting participants who did not want to fill out the survey could still look like they were participating; in this way, we preserved the privacy of the respondents.

Response rates could only be calculated for faculty in the 8 CoE departments, because it was unknown how many staff from each department typically attend a departmental faculty meeting and therefore had the opportunity to respond. Faculty response rates by department ranged from 42% to 80%. Overall, 115 faculty and 30 staff (with 9 people not reporting their employment category) responded to the survey, for a 61% overall faculty response rate. The demographic characteristics of respondents are reported in Table 1.

Table 1.

Demographic Characteristics of Survey Respondents

Characteristic N % # Missing
Female 38 25.7% 7
Male 109 70.8%
Persons of Color 37 24.0% 9
Majority Persons 108 70.1%
Faculty 115 74.7% 9
Staff 30 19.5%
Attended Workshop 121 78.6% 8
Did Not Attend Workshop 25 16.2%

Note: “Persons of Color” are respondents who self-identified as either “non-white” or “faculty/staff of color.” “Majority persons” did not identify as nonwhite or a person of color.

3. Results

3.1. Individual Gains from Workshop

3.1.1. Reactions of individuals.

Evaluation data and interview data showed that most workshop attendees had a positive reaction to the workshop content. A majority, 71.1%, of workshop participants who filled out the evaluation form rated the workshop overall as “very valuable”; only 2.2% said that it was not at all valuable; the remainder said it was “somewhat valuable.” Almost all, 97.0%, would recommend the workshop to a colleague.

When we asked interviewees in person what they thought about the workshop, most had an overall positive reaction. They found it useful and the information interesting. Female interviewees noted in particular that they saw their colleagues listening to the content:

“It was very nice to see your colleagues actually sit and listen. And I think that’s the least that you can ask for them to do. It was definitely that was something made me very happy.”

(Diane, tenured professor)

“[I’m] a little bit more willing to say something because I feel like, ‘Okay, now you guys have heard this stuff from someone other than me.’”

(Gabriella, tenured professor)

Some workshop participants did not enjoy the workshop and resented the time they spent discussing issues of implicit bias. Some of those who felt the workshop was a “waste of time” indicated that they already knew a great deal about implicit bias:

“… especially if somebody had done it before, who are like ‘ok I’ve done that I understand that and I’m not feeling the need for another version’…”

(Ryan, tenured professor)

But others who felt it wasted their time were resistant to the topic. Reporting on their colleagues, two faculty explained:

“…the people who need it the most are probably the most resistant to taking in that information and believing that there’s actually something there.”

(Gabriella, tenured professor)

“The senior faculty, some of them are like, ‘Uh, why do I have to go through this?’”

(Joshua, tenured professor)

3.1.2. Individual knowledge gains.

Workshop evaluations conducted immediately after the workshop show that participants self-reported that they gained knowledge about every concept we cover in the workshop, as shown in Table 2. (Participants rated their knowledge before and after the workshop in two separate items completed at the end of the workshop.) “Stereotype Threat” and “Stereotype Priming” showed the largest gains in concept knowledge, and “Using Diversity Affirming Primes” and “Act to Combat Microaggressions” showed the largest gains in knowledge about bias-reducing strategies. All knowledge gains were statistically significant at p<0.001 using a paired-sample t-test.

Table 2.

Knowledge Gain in Implicit Bias Concepts, Before and After Workshop

Bias Concepts Knowledge BEFORE Mean (SD) Knowledge AFTER Mean (SD) Knowledge Gain
Expectancy Bias 1.45 (0.77) 2.43 (0.51) +0.98***
Competency Bias 1.48 (0.76) 2.47 (0.53) +0.98***
Stereotype Priming 1.20 (0.83) 2.36 (0.60) +1.16***
Stereotype Threat 1.21 (0.82) 2.39 (0.57) +1.18***
Microaggressions 1.38 (0.86) 2.48 (0.53) +1.10***

Note: Responses on a 4-point scale from 0=No knowledge to 3=Much knowledge. SD=Standard Deviation.

***

Paired-sample t-test, p<0.001

At the end of the workshop, participants were asked to commit to practicing one or more of the bias-reducing strategies we suggested in the workshop. As shown in Table 3, “acting to reduce microaggressions” and “individuate” were the top strategies mentioned by participants. Individuation refers to the conscious practice of ensuring that you see each person you encounter as an individual, distinct or unique from any social category (e.g., gender, race, ethnicity, religion) to which they belong. See the Supplemental Materials for definitions of all workshop concepts.

Table 3.

Knowledge Gain in Strategies to Reduce Implicit Bias in Behaviors, Before and After Workshop

Bias Reduction Strategies Knowledge BEFORE Mean (SD) Knowledge AFTER Mean (SD) Knowledge Gain
Recognize, Label, and Challenge Stereotypes 1.58 (0.71) 2.43 (0.59) +0.84***
Individuate and Perceive Variability 1.49 (0.78) 2.45 (0.58) +0.96***
Implement Evaluation Practices that Minimize Bias 1.50 (0.81) 2.39 (0.58) +0.89***
Use Diversity Affirming Primes 1.12 (0.84) 2.31 (0.63) +1.19***
Foster Growth Mindsets 1.50 (0.90) 2.46 (0.58) +0.96***
Take the Perspective of Others 1.94 (0.77) 2.56 (0.55) +0.63***
Act to Combat Microaggressions 1.23 (0.85) 2.38 (0.65) +1.15***

Note: Responses on a 4-point scale from 0=No knowledge to 3=Much knowledge. SD=Standard Deviation

***

Paired-sample t-test, p<0.001

3.1.3. Individual Stages of Change.

As outlined in our theoretical framework, we were interested in tracking individuals’ progress through four major stages of change as they move from unconscious incompetence to unconscious competence: awareness, motivation, self-efficacy, and action.

Survey data collected from department members 3–12 months after the department received a workshop showed that workshop attendees were more likely to be aware of implicit bias in the environment around them, compared with people who did not attend the workshop. As Table 4 shows, compared to those who did not attend the workshops, workshop attendees were more likely to notice when others exhibit bias towards others, and more likely to say that individuals from minority groups were not too sensitive about unintended offenses.

Table 4.

Awareness of Bias

Attended Workshop
Awareness Item b SE
I notice when others exhibit bias towards individuals from minority groups 1.06*** 0.23
I could unintentionally behave in biased ways towards individuals from minority groups 0.38 0.36
I consider discrimination against individuals from minority groups to be a serious social problem −0.02 0.16
Individuals from minority groups are overly sensitive about unintended offenses (reverse-coded) 0.68* 0.32
Jokes that rely on stereotypes are offensive. −0.49 0.26
Stereotyping is harmless (reverse-coded) 0.19 0.12

Notes: N=139. Ordinary Least Squares (OLS) Regression, b=OLS coefficient and SE=Standard Error for a dummy variable indicating workshop attendance or not. Models control for: department, female, Non-White, faculty. Additional analyses included: OLS regression without department dummies, regression with cluster (department) Standard Errors controlled, and multi-level modeling. Results were similar across models, available upon request.

*

p<0.05;

***

p<0.001

We were disappointed to see that workshop attendees were slightly less likely to agree that “jokes that rely on stereotypes are offensive” compared to their colleagues who did not attend the workshop, although this was not a statistically significant difference. We would have hoped that workshop attendees agreed more that such jokes were offensive after the workshop. Similarly unfortunately, we found very little change in the motivation, self-efficacy, positive outcomes associations, and action associated with behavioral change towards “Breaking the Bias Habit®1.”

In our interview data, we also did not detect major changes in personal actions to reduce implicit bias, although interviewees sometimes reported changes they saw in the behavior of others:

“I don’t know how much we’ll actually make people change. But at least they heard, and they were listening, or sometimes engaged. And that definitely is a good thing.”

(Diane, tenured professor)

“[I talked] a little bit, informally, just with some [of] my [research colleagues]. I think that in general the sentiments were similar, in that there was a very positive feeling about this workshop. It was useful information.”

(Kevin, untenured professor)

“The junior faculty seem a lot more interested [than the senior faculty].”

(Joshua, tenured professor)

3.2. Changes in Departmental Climate

3.2.1. Overall ratings of climate.

The engineering faculty and staff we talked to had consistently good things to say about the climate in their departments:

“It is an open, welcoming climate, I think.”

(Ryan, tenured professor)

“I would say overall it’s pretty welcoming overall. There’s always individuals…”

(Gabriella, tenured professor)

“I never felt disrespected. That’s for sure.”

(Diane, tenured professor)

At the same time, many interviewees described the climate as “business like,” which did not have consistently positive connotations:

“I would say [the climate is] so-so. But it’s because everybody’s so busy….so there’s not much chance to feel welcoming or rejected, you know.”

(Charles, tenured professor)

“I feel valued and respected…uh, a bit. I mean you know it’s a university, there’s all these you know raging egos running around, and engineering is less bad than most parts of it. But everybody is just focused on doing what they as individuals do.”

(Eric, tenured professor)

“I don’t see a lot of my department like on a daily basis, except for monthly faculty meetings. So we don’t spend enough time together to get on each other’s nerves maybe [laughs]. But yeah, overall, I’ve felt support from the department.”

(Paul, untenured professor)

From this relatively positive baseline, we asked respondents whether they felt their department climate improved as a result of the workshops. Some respondents noted changes to the physical environments in the department, especially the department websites:

“But I know, again, when members of [the chair’s office] went back and looked at our website and some other elements…we found some examples or some words or some presentations that could be considered [biased], based on what we learned from the workshop. And it was certainly implicit, meaning that it was unintentional and we really didn’t realize we were doing it. … We made those appropriate fixes.”

(Lucas, tenured professor)

Some faculty noted a concerted effort to interrupt women less:

“I had been sensitive to this in the past…, but now it really stands out, like in a meeting if a male colleague starts to speak over a female colleague, I notice that. … So that’s an example, it still happens sometimes I think people are being more aware of that and trying to self-counteract and people are intervening using some of the strategies…”

(Ryan, tenured professor)

“I also see people checking themselves when they interrupt, and I think they may remember the workshop, but they definitely will realize they’ve interrupted somebody and apologize for it. So, I think there is a reflection and a, I think, fairly persistent influence of the workshops.”

(Oscar, tenured professor)

Some interview respondents perceived changes to faculty hiring processes, and/or graduate student recruitment practices:

“Actually at the last department meeting we talked about the issue of bias when we’re recruiting students. And you know we were at this website where all the applications from the students are coming in. And we looked at them and we say oh we like that person, we don’t like that person. … We talked about our implicit bias when we’re actually looking at their dossier.”

(Ivan, untenured professor)

“I know one of the things that we changed quite a bit was in our hiring process in the [position descriptions] that we put out, we changed a fair amount of the language to, I think some of it was advice provided to us as part of the workshop or something thereafter, but as well as how some of the [position description] was presented, made it implicitly or potentially interpreted as implicit bias. So that led to a fair amount of editing with that document.”

(Lucas, tenured professor)

And some simply perceived a slight positive change as a result of more awareness of implicit bias:

“…back to the climate thing. It just feels like it’s more positive and in terms of just trying to be more sensitive. I really think that the facts that were brought up during the workshop were ah-ha moments, for everyone. And I think people have really been thinking about those.”

(Felicia, staff)

The quantitative survey data we collected supported this general feeling of slight improvements in department climate in the College of Engineering. We asked respondents to report whether five measures indicating an inclusive department climate had become more or less common since the workshop was offered in their departments. If a change was negative it was coded −1; if no change occurred, the code was zero; and if a positive change occurred on the measure, a code of +1 was assigned. Thus, a positive value for these items in Tables 5 through 7 indicate a positive change in the department on the given measure.

Table 5.

Department Climate Change

Climate change item Mean SD SE
Respectful interactions in department meetings 0.24*** 0.51 0.05
Inclusive physical environments in common department areas 0.18*** 0.44 0.05
Jokes or sarcastic comments about diversity and inclusion 0.31*** 0.57 0.06
Spotlighting or targeting of women, minorities, or other individuals from minority groups 0.07 0.53 0.05
Explicit discussions of potential biases in department processes (e.g., admissions, hiring, promotion, awards) 0.66*** 0.54 0.05

Notes: N=100. Means are calculated on a three-point scale, with +1 indicating a positive change, −1 indicating a negative change, and 0 indicating no change. One-sample t-test; null hypothesis mean=0.

***

p<0.001

Except for the issue of “spotlighting” underrepresented persons in the department (highlighting or objectifying somebody’s identity status as a spokesperson for a broad category—for example, when a student is called on in class and asked “what do Latinas think about this issue?”), all of the climate measures we asked about showed significant improvement since the workshop, as shown in Table 5. For example, most faculty felt that “respectful interactions in department meetings” had become more common in the department compared to before the workshop.

However heartening these findings might be, it is important to ask whether underrepresented persons—women, faculty/staff of color—also see positive changes in the department. To discover this, we performed two-sample t-tests to test whether men vs. women, and persons of color vs. majority persons, differentially saw changes in the department after the workshop was implemented. On some measures, only men/majority people saw the positive change, and the underrepresented group saw less or no change.

As shown in Table 6, women reported no change (or, an equal amount of both positive and negative change) in the areas of both “respectful interactions in department meetings” and “jokes or sarcastic comments about diversity and inclusion.” In the other areas of department climate, they experienced positive changes similar to the men. In general, perhaps because they experience it directly, women and persons of color tend to be more perceptive of negative aspects of climate than their men/majority colleagues (Sheridan et al., 2007; WISELI, 2019), as recognized by one faculty member who reported only hearing about climate “awareness” from his female colleagues:

Table 6.

Department Climate Change, By Gender

Women Men
Climate change item Mean SE Mean SE
Respectful interactions in department meetings −0.05c 0.09 0.31a,c 0.06
Inclusive physical environments in common department areas 0.11 0.07 0.20a 0.06
Jokes or sarcastic comments about diversity and inclusion 0.00c 0.10 0.39a,c 0.07
Spotlighting or targeting of women, minorities, or other individuals from minority groups 0.10 0.10 0.08 0.06
Explicit discussions of potential biases in department processes (e.g., admissions, hiring, promotion, awards) 0.65a 0.13 0.66a 0.06

Notes: Women N=20; Men N=80. Means are calculated on a three-point scale, with +1 indicating a positive change, −1 indicating a negative change, and 0 indicating no change. One-sample t-test; null hypothesis mean=0 tests for change within a group; two-sample t-test tests for differences between groups.

a

p<0.001 within-group;

b

p<0.01 within-group;

c

p<0.01 between-group.

“Okay, so I’m in a department that’s very traditional in the sense that our distribution between males and the female faculty members… is very one-sided. And the awareness that I’ve heard of [department climate] is almost universally coming from my discussions with the female faculty members. … I think there’s more awareness among the minority individuals in this case than the majority individuals just because I’ve heard more comments from them than I’ve ever heard from the others.”

(Brian, tenured professor)

Table 7 shows that faculty/staff of color reported the same positive change on all items except the last, “explicit discussions of potential biases in department processes.” Here, persons of color felt there was some positive change, but not nearly as much as majority persons reported.

Table 7.

Department Climate Change, by Race/Ethnicity

Persons of Color Majority
Climate change item Mean SE Mean SE
Respectful interactions in department meetings 0.12 0.11 0.26a 0.06
Inclusive physical environments in common department areas 0.13 0.11 0.19a 0.05
Jokes or sarcastic comments about diversity and inclusion 0.22 0.11 0.33a 0.07
Spotlighting or targeting of women, minorities, or other individuals from minority groups 0.11 0.10 0.06 0.07
Explicit discussions of potential biases in department processes (e.g., admissions, hiring, promotion, awards) 0.37b,c 0.11 0.76a,c 0.06

Notes: Persons of color N=27; Majority Persons N=72. “Persons of Color” are respondents who self-identified as either “non-white” or “faculty/staff of color.” “Majority persons” did not identify as nonwhite or a person of color. Means are calculated on a three-point scale, with +1 indicating a positive change, −1 indicating a negative change, and 0 indicating no change. One-sample t-test; null hypothesis mean=0 tests for change within a group; two-sample t-test tests for differences between groups.

a

p<0.001 within-group;

b

p<0.01 within-group;

c

p<0.01 between-group.

4. Limitations

The methods used in this study to ascertain the effectiveness of the “Breaking the Bias Habit” workshops in the College of Engineering had several limitations. First, we only looked within the College of Engineering for change. We did not employ a control group for this study, nor did we create experimental conditions within the College of Engineering to look for change that can be directly attributed to the workshop. Second, we did not employ any pre/post design studies for this analysis. All data for this study were collected after a department (and individuals in that department) participated in the workshop. We did ask some retrospective questions after workshop participation (e.g., how much knowledge of a concept did you have before a workshop compared to after?), but this is not true pre/post data. Future analysis using a campus-wide survey, the Study of Faculty Worklife (WISELI, 2019), may address some of these limitations, as these data allow us to not only compare department climate outside of the CoE with climate within Engineering over this time period, but we can also use pre/post measures on several climate items to look for change, because this survey has been administered approximately every 4 years since 2003. Finally, we must acknowledge the lack of intersectional work in this study. We were unable to look for specific experiences of women/men of color relative to majority men and women, or experiences within specific racial/ethnic groups, due to the very small number of women of color in the College of Engineering overall. We made every effort to include men and women of color in our interviews, but disaggregating by gender within that category would have made our respondents far too identifiable.

5. Discussion

The intervention we devised, “Breaking the Bias Habit®: A Workshop to Promote a Diverse, Welcoming, and Inclusive Campus,” does not appear alone to move individual faculty through the stages of change as we had hoped. The survey, with more than a 60% faculty response rate, did not find that faculty who participated in the workshop improved their personal motivation to act without bias, their self-efficacy to act, nor their self-reported actions.

While this is a disappointing finding, we did find many positive outcomes from our workshop intervention that will move the College of Engineering towards creating a more inclusive environment. Faculty who attended the workshop had significant increases in their awareness of implicit bias in their workplaces, a finding that appeared in the survey data as well as in our interviews with CoE faculty. While awareness alone is not enough to promote behavioral change and, in fact, may inhibit such change (Duguid and Thomas-Hunt, 2015), it is a necessary first step (Carnes et al., 2005; Prochaska and DiClemente, 1983). Interview comments suggest that the workshops may have opened important conversations about bias in these departments.

One reason we may not have been able to measure the individual-level motivation, self-efficacy, and action changes we anticipated was that a much longer time-period passed between workshop attendance and survey measurement than in previous studies. In our study, the shortest amount of time between workshop and survey was 6 months, and the longest time was 12 months. In our previous work (upon which this study was based), we saw the motivation, self-efficacy, and self-reported action changes at 3 months (Carnes et al., 2015). Perhaps having such a long time between workshop and measurement dilutes the direct impact of the workshop because so many other factors impact behavior over time.

At the same time, our short-term measures, gathered from evaluation forms directly after the workshop, indicate that at least some self-efficacy was gained, in that the attendees’ knowledge of bias-reduction strategies increased significantly after workshop participation. In particular, faculty gained the most knowledge about “Using Diversity Affirming Primes” and “Act to Combat Microaggressions.” Indeed, in the interviews we conducted with faculty, respondents specifically mentioned both of these strategies at play in their department (e.g., updating websites to be more inclusive is an example of “Using Diversity Affirming Primes”, and ensuring that women do not get interrupted in faculty meetings would be “Act to Combat Microaggressions.”)

At the department level, we were excited to see evidence of more positive change than we were able to measure at the individual level. Four of our five measures of departmental climate improved significantly overall. Some of these measures, such as improvements in respectful interactions and explicit discussions of potential biases in departmental processes, were also mentioned specifically by our interview participants as positive changes they had seen in their departments since the participation in the Breaking the Bias Habit® workshop. Importantly, women faculty and faculty of color in our CoE departments did not show significant decreases in their experiences of climate in the approximately 6–12 months after the workshop. It is always a concern that delivering a workshop of this nature to a departmental unit, which may have very few underrepresented persons in the workshop, could be a negative experience for those few, especially if the majority colleagues do not take the experience seriously and make jokes or increase their biased behaviors afterwards. Fortunately, women faculty and faculty of color did not report increases in joking, sarcastic comments, or spotlighting/targeting after the workshop. While it is true that the overall improvements reported in our survey seem to be driven by male/majority faculty responses, for the item “explicit discussions of potential biases in department processes,” both women and faculty of color reported significant increases in these activities in their departments.

For departments, schools/colleges, and/or universities interested in adopting a program of implicit bias education for their faculty and staff, we offer some advice. First, we highly recommend conducting workshops of this type within a work unit (such as a department), particularly if a primary goal is to positively impact department climate. As our previous study (Carnes et al., 2015) and data from this study support, we achieve the cultural change we desire in a unit when a critical mass (e.g., more than 25%) of the unit participates together in a training, so that both individual and collective behavioral change can reinforce each other. Second, we highly recommend that participation in such training not be mandatory. This may seem contradictory, given our previous advice to deliver these workshops within units, but studies show that coercing people to participate in diversity-related education can backfire (Kawakami et al., 2007; Legault et al., 2011; Dobbin and Kalev, 2016). The approach taken in the CoE was to require all faculty to participate in some kind of diversity-related training, but not necessarily the “Breaking the Bias Habit®” workshops outlined here. That is, faculty and staff in the CoE could complete this requirement in any number of ways; they were not forced to participate in our workshop. This created a much more collegial environment within the workshop, as participants felt they had chosen voluntarily to be there. Third, we recommend creating a workshop presenter team that is as diverse as possible. Our team of workshop presenters consisted of all women (a disadvantage; we received several comments that it would be better if a man was also a presenter), but included women with diverse racial and ethnic heritages, which increased participant receptivity. Finally, any effort to conduct diversity-related programming within an academic unit should strive to ensure the safety of persons with underrepresented identities who participate in those workshops. Ensuring that pilot audiences are as diverse as possible; intentionally gathering feedback from underrepresented gender, racial/ethnic, sexual orientation, and dis/ability identities to ensure that the learning environment is safe; investigating workshop outcomes for these underrepresented groups and not just the majority is crucial to ensuring that climate issues for persons in such groups were not exacerbated as a result of the workshop.

As a result of the positive results reported here (and the lack of evidence of doing harm for our women faculty and faculty of color in the CoE), we are continuing to offer these “Breaking the Bias Habit®” workshops within the CoE and across the UW-Madison campus more broadly. We are developing versions of the workshop to deliver to CoE undergraduate and graduate students, as well as a slightly altered version that works well with non-faculty staff at the university. In future work, we will use campus-wide climate surveys to more fully measure department climate change over time, and will also look at hiring patterns in the College of Engineering, to see if the self-reported process changes reported in our interviews have made a difference in that area. In short, we see positive impacts of implicit bias education for university faculty and we are continuing to implement these trainings and measure their outcomes, as we strive for a more diverse, inclusive and welcoming environment for all members of our university community.

Supplementary Material

Supplementary Material

Acknowledgments

This research was supported by the College of Engineering and the Office of the Chancellor, University of Wisconsin-Madison. Dr. Carnes’s research on diversifying the scientific workforce is funded by NIH R35GM122557.

Footnotes

1

Results available upon request.

6. References

  1. Bandura A (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. doi: 10.1037/0033-295X.84.2.191. [DOI] [PubMed] [Google Scholar]
  2. Bilimoria D, & Liang X (2012). Gender equity in science and engineering: Advancing change in higher education. New York: Routledge. Retrieved from http://www.routledge.com/books/details/9780415885621/. [Google Scholar]
  3. Braun V, & Clarke V (2012). Thematic Analysis. In Cooper H (Ed.), APA handbook of research methods in psychology: vol. 2. Research designs Washington, DC: American Psychological Association. doi: 10.1037/13620-000. [DOI] [Google Scholar]
  4. Butkus R, Serchen J, Moyer DV, Bornstein SS, Hingle ST, & for the Health and Public Policy Committee of the American College of Physicians. (2018). Achieving gender equity in physician compensation and career advancement: A position paper of the American College of Physicians. Annals of Internal Medicine, 168(10), 721. doi: 10.7326/M17-3438. [DOI] [PubMed] [Google Scholar]
  5. Carnes M, Devine PG, Isaac C, Baier Manwell L, Ford CE, Byars-Winston A, Fine E, & Sheridan J (2012). Promoting institutional change through bias literacy. Journal of Diversity in Higher Education, 5(2), 63–77. doi: 10.1037/a0028128. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Carnes M, Devine PG, Manwell LB, Byars-Winston A, Fine E, Ford CE, Forscher P, Isaac C, Kaatz A, Magua W, Palta M, & Sheridan J (2015). Effect of an intervention to break the gender bias habit for faculty at one institution: A cluster randomized, controlled trial. Academic Medicine, 90(2), 221–230. doi: 10.1097/ACM.0000000000000552. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Carnes M, Handelsman J, & Sheridan J (2005). Diversity in academic medicine: The stages of change model. Journal of Women’s Health, 14(6), 471–475. doi: 10.1089/jwh.2005.14.471. [DOI] [PubMed] [Google Scholar]
  8. Centola DM (2013). Homophily, networks, and critical mass: Solving the start-up problem in large group collective action. Rationality and Society, 25(1), 3–40. doi: 10.1177/1043463112473734. [DOI] [Google Scholar]
  9. Devine PG (1989). Stereotypes and prejudice: Their automatic and controlled components. Journal of Personality and Social Psychology, 56(1), 5–18. doi: 10.1037/0022-3514.56.1.5. [DOI] [Google Scholar]
  10. Devine PG, Forscher PS, Cox WTL, Kaatz A, Sheridan J, & Carnes M (2017). A gender bias habit-breaking intervention led to increased hiring of female faculty in STEMM departments. Journal of Experimental Social Psychology, 73, 211–215. doi: 10.1016/j.jesp.2017.07.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Dobbin F, & Kalev A (2016). Why diversity programs fail. Harvard Business Review, (Jul-Aug). Retrieved from https://hbr.org/2016/07/why-diversity-programs-fail. [Google Scholar]
  12. Duguid MM, & Thomas-Hunt MC (2015). Condoning stereotyping? How awareness of stereotyping prevalence impacts expression of stereotypes. Journal of Applied Psychology, 100(2), 343–359. doi: 10.1037/a0037908. [DOI] [PubMed] [Google Scholar]
  13. Fine E, Sheridan J, Bell CF, Carnes M, Neimeko CJ, & Romero M (2018). Teaching academics about microaggressions: A workshop model adaptable to various audiences. Understanding Interventions Journal, 9(2). https://www.understandinginterventionsjournal.org/issue/1000. [Google Scholar]
  14. Forscher PS (2016). The individually-targeted habit-breaking intervention and group-level change (Doctoral dissertation). University of Wisconsin-Madison. [Google Scholar]
  15. Forscher PS, Mitamura C, Dix EL, Cox WTL, & Devine PG (2017). Breaking the prejudice habit: Mechanisms, timecourse, and longevity. Journal of Experimental Social Psychology, 72(Supplement C), 133–146. doi: 10.1016/j.jesp.2017.04.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Greenwald AG, McGhee DE, & Schwartz JLK (1998). Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74(6), 1464–1480. doi: 10.1037/0022-3514.74.6.1464. [DOI] [PubMed] [Google Scholar]
  17. Howell WS (1982). The empathic communicator. Los Angeles, CA: Sage. [Google Scholar]
  18. Kawakami K, Dovidio JF, & van Kamp S (2007). The Impact of counterstereotypic training and related correction processes on the application of stereotypes . Group Processes & Intergroup Relations, 10(2), 139–156. doi: 10.1177/1368430207074725. [DOI] [Google Scholar]
  19. Legault L, Gutsell JN, & Inzlicht M (2011). Ironic effects of antiprejudice messages: How motivational interventions can reduce (but also increase) prejudice. Psychological Science, 22(12):1472–1477. doi: 10.1177/0956797611427918. [DOI] [PubMed] [Google Scholar]
  20. Nonaka I (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1). doi: 10.1287/orsc.5.1.14. [DOI] [Google Scholar]
  21. Paluck EL, & Shepherd H (2012). The salience of social referents: A field experiment on collective norms and harassment behavior in a school social network. Journal of Personality and Social Psychology, 103(6), 899–915. doi: 10.1037/a0030015. [DOI] [PubMed] [Google Scholar]
  22. Prochaska JO, & DiClemente CC (1983). Stages and processes of self-change of smoking: Toward an integrative model of change. Journal of Consulting and Clinical Psychology, 51(3), 390–395. doi: 10.1037/0022-006X.51.3.390. [DOI] [PubMed] [Google Scholar]
  23. Prochaska JO, & Velicer WF (1997). The transtheoretical model of health behavior change. American Journal of Health Promotion, 12(1), 38–48. doi: 10.4278/0890-1171-12.1.38. [DOI] [PubMed] [Google Scholar]
  24. Sevo R, & Chubin DE (2008). Bias literacy: A review of concepts in research on discrimination. Washington, DC: American Association for the Advancement of Science Center for Science & Engineering Capacity. [Google Scholar]
  25. Sheridan J, Fine E, Pribbenow CM, Handelsman J, & Carnes M (2010). Searching for excellence & diversity: Increasing the hiring of women faculty at one academic medical center. Academic Medicine, 85(6), 999–1007. doi: 10.1097/ACM.0b013e3181dbf75a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Sheridan J, Pribbenow CM, Fine E, Handelsman J, & Carnes M (2007). Climate change at the University of Wisconsin-Madison: What changed, and did ADVANCE have an impact?” Women in ENgineering Programs & Advocates Network (WEPAN) 2007 Conference Proceedings. https://journals.psu.edu/wepan/article/view/58488/58176. [Google Scholar]
  27. Sue DW (2010). Microaggressions in everyday life: Race, gender, and sexual orientation. Hoboken, NJ: John Wiley & Sons, Inc. [Google Scholar]
  28. Stewart AJ, Malley JE, & LaVaque-Manty D (2007). Transforming science and engineering: Advancing academic women. Ann Arbor, MI: University of Michigan Press. Retrieved from http://madcat.library.wisc.edu/cgi-bin/Pwebrecon.cgi?BBID=7443490. [Google Scholar]
  29. Stewart AJ, & Valian V (2018). An inclusive academy: Achieving diversity and excellence. Cambridge, MA: MIT Press. [Google Scholar]
  30. Women in Science & Engineering Leadership Institute (WISELI). (2019). Study of faculty worklife at UW-Madison. Accessed November 19, 2020; retrieved from https://wiseli.wisc.edu/research/sfw/.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Material

RESOURCES