Lessons Learned from Implementing Unconscious Bias Training at an Academic Medical Center

Abstract

Introduction

In 2018, our Midwestern university medical center began offering unconscious bias training. Each session concluded with a standard evaluation. We analyzed two years of data that focused on three areas: 1) whether demographic differences or amount of prior knowledge on the topic influenced the training experience; 2) what participants liked best about the training; and 3) whether participants’ stated intentions to apply their learning at the end of the training aligned with institutional goals of the training.

Methods

Participants attended sessions open to the campus community pre-scheduled by the Office for Diversity, Equity, and Inclusion and posted on its website. Chi-square tests were utilized to test associations between outcomes and questionnaire responses. Outcome measures included race/ethnicity, prior knowledge level, and overall rating of the training. Thematic analysis was used to code comments and establish themes from two open-ended survey questions.

Results

Significant differences were found by race and ethnicity for all questionnaire responses; each were p < 0.001. Those who reported they had advanced/expert knowledge on the topic were less likely to report the training increased their knowledge, and those who reported their race as White/Caucasian tended to give the training the highest overall rating, as did heterosexuals. Through thematic analysis, participants valued the interactive nature of the training sessions, the use of storytelling, and the safety of the learning environment. Participants’ intention to apply their learning indicated they had gained general awareness of bias and settings where it might influence their work.

Conclusions

In an effort to foster a better working and learning environment for those who are underrepresented in the health professions, training was provide that may not have met the expectations of all participants. At the same time, participants who identified as White clearly increased their awareness of bias. Therefore, it is recommended to move away from one-size-fits-all unconscious bias training and develop a robust training continuum to provide ongoing advancement for diverse audiences.

INTRODUCTION

Unconscious bias (UB) refers to mental shortcuts we take in making decisions.1 These shortcuts typically are unexamined and outside our awareness. Without meaning to, some individuals might perpetuate discriminatory behavior.2 Some aspects of discrimination are so deeply ingrained in society that certain groups, such as people who have disabilities or people of color, collectively often have interactions involving automatic and often harmful associations.3

Impacts of UB have been documented extensively in academic medicine and patient care. Areas such as the recruitment, hiring, promotion, and retention of Black, Indigenous, and People of Color (BIPOC), LGBTQIA+, and female faculty of color,4–7 along with admission, retention, and graduation rates of underrepresented students,8–10 all have been linked to UB. Increasingly, UB is used as a concept to explain, and at times justify, lack of diversity in faculty hiring, student admissions, and curricular content.11 Like the general population, healthcare providers have been shown to hold bias and stereotypes against patients from marginalized and minoritized groups already burdened by health disparities. UB research documents treatment differences in pain management, chronic disease management, and psychiatric care among others.12,13 Health care providers who hold negative unconscious biases against Black/African American and Hispanic patients are more likely to identify these patients as non-compliant or uncooperative.12–14 Conversely, providers tended to favor patients specified as White in their clinical judgment and behavior.15,16

UB trainings have emerged as a reactive and ubiquitous strategy in many organizations to address bias and promote diversity, equity, and inclusion (DEI). Although previous research has indicated that similar training effectively influences participants’ awareness of different biases,17,18 critics argue that UB training is an insufficient response to institutional inequities,2,19 and little evidence exists on the ability of UB training to effect institutional change.20 A recent meta-analysis of 492 studies and 87,418 participants found the effects of such training to be relatively weak and rarely translates into change of explicit measures in behavior.21 Moreover, some UB training interventions may lead to further entrenchment of biases.20,21

Despite these flaws, it remains imperative to educate people in academic medicine about the automatic cognitive processes that impact important decisions and judgments in their work. While UB training may be a contested educational method, these cognitive processes are well documented and extensively researched in the work of Daniel Kahneman and others.22,23 Thus, in 2018, amidst rising concern of the impact of UB on internal policies and practices, our Midwestern medical center began offering UB training to the campus community. Since then, more than 2,500 students, staff, faculty, residents, and others from our schools of medicine, nursing, pharmacy, and health professions have participated in this training.

As our university medical center and other institutions consider ways to address flaws in diversity programming, it is of paramount importance to identify effective teaching strategies for delivering these difficult topics. Developing a deeper understanding of what works to bring about behavioral change (and what does not) is also critical. Two years of evaluation data from our UB training were evaluated with three questions in mind: Did a one-size-fits-all training meet the needs and expectations of participants; did participants’ intentions to apply learning align with the goals of the training; and did participant feedback indicate key instructional strategies to emphasize as we consider next steps in DEI training?

METHODS

Training Materials

The university medical center sent 16 faculty and staff to a four-day train-the-trainer course endorsed by the American Association of Medical Colleges. Trainees became certified to facilitate a two-hour training on unconscious bias using materials provided (slides and a participant guide). The training emphasized bias as an unconscious process of the mind and included activities to help people become more aware of these mental shortcuts. It also emphasized documented impacts of bias in healthcare and provided some strategies to encourage individual growth.

Participants

Trainings were offered as general sessions open to any interested employee and advertised via broadcast emails and announcements. Supervisors also were encouraged to request training for their teams or department. From 2018–2020, 1,408 employees, including supervisors, registered for general sessions. Training was provided 54 times to teams, departments, and committees by trained facilitators reaching an estimated 2,500 individuals across the medical center. At the end of each training, participants were asked to complete anonymously a standard questionnaire involving demographic information, participants’ experience during training, need for additional resources, and intentions to apply the content learned. Surveys included both multiple-choice and open-response fields. A total of 923 participants completed these end-of-training questionnaires.

Questionnaire

A 23-item questionnaire was developed to evaluate participant perception of the Unconscious Bias training. We used questionnaire responses to: 1) evaluate demographic differences of respondents; 2) assess differences in perception by race and ethnicity; 3) measure the impact of prior knowledge on training satisfaction; 4) identify training areas that were most likely to have lower verses higher ratings; and 5) qualitatively determine trends in satisfaction and plans to utilize what was learned.

Data Analysis Plan

This project was deemed and approved as a quality improvement study by the university’s institutional review board. Descriptive statistics were used to summarize responses to the UB training questionnaire. Frequencies and percentages were used for categorical items. Chi-square tests were utilized to test associations between each outcome and questionnaire responses. Where data were sparse, Fisher’s exact test of association was used. In each case, two-sided tests were conducted using an alpha value adjusted for multiple tests, which resulted in p = 0.001 as level of significance. Analyses were conducted in IBM SPSS Statistics, version 26.

Open item responses were analyzed by two team members with backgrounds in education, sociology, and communications. Thematic analysis, a method often employed for analyzing qualitative data by identifying, analyzing, and reporting repeated patterns, was used to examine the response to two questions: 1) What did you like best about this training? and 2) In which ways are you planning to use what you learned today? Braun and Clarke’s six-stage analysis process was used to identify patterns or themes within the narrative data.24 Open item responses were inductively coded by two coders. The researchers engaged in an independent first reading, becoming familiar with the data by reading and re-reading the responses and noting initial impressions. Coders then discussed initial emergent topics and themes. Inconsistencies were resolved before proceeding with the coding process. The analysis process was iterative and emergent themes were cross questioned and critiqued by coders, clarified, defined, and reviewed until saturation was reached.25 Results were summarized, which included compelling extracted examples.

RESULTS

Respondents

Of the 2,500 attendees, 923 completed the questionnaire for a response rate of about 37%. Table 1 summarizes the responses to the demographic questions. Most respondents were female (n = 554; 74%), described their sexual orientation as heterosexual (n = 677; 82%), had not served in the military (n = 805; 94%), did not have a disability (n = 789; 92%), and were self-described as university medical center staff (n = 445; 53%). Over half of respondents reported they had intermediate prior knowledge of UB prior to the training event (n = 447; 54%).

Table 1.

Summary of participant responses to demographics.

Description	Missing	Frequency (n)	%
Total participants		923	100.0
What is your age?	57
18 – 24 years		86	9.9
25 – 34 years		257	29.7
35 – 44 years		194	22.4
45 – 54 years		154	17.8
55 years +		145	16.7
Prefer not to answer		30	3.5
What is your racial or ethnic identification?	62
White/Caucasian		650	70.4
Black/African American		55	6.0
Hispanic/Latino		43	4.7
Asian, American Indian/Alaskan Native, Pacific Islander		38	4.1
Other, Multi-racial		36	3.9
Prefer not to answer		39	4.2
What is your gender identity?	171
Male		176	23.4
Female		554	73.7
Other		22	2.9
Which of the following best describes your sexual orientation?	97
Asexual		41	5.0
Bisexual		19	2.3
Gay		11	1.3
Heterosexual		677	82.0
Lesbian		8	1.0
Other*		13	1.4
Prefer not to answer		57	6.9
Have you ever served in any branch of the Unites States military?	64
Yes		34	4.0
No		805	93.7
Prefer not to answer		20	2.3
Do you currently have a diagnosed disability?	69
Yes		29	3.4
No		789	92.4
Prefer not to answer		36	4.2
Which of the following best describes your PRIMARY affiliation?	79
Faculty		144	17.1
Resident/Fellow		55	6.5
Student		118	14.0
Hospital Staff		41	4.9
University Medical Center Staff		445	52.7
Community Member		5	0.6
Other		36	4.3
If you are a student, what is your academic program?	769
School of Nursing		6	3.9
School of Medicine		36	23.4
School of Health Professions		97	63.0
Graduate Studies		15	9.7
If you are a student, are you an international student?	740
Yes		11	6.0
No		172	94.0
What was your knowledge of the topic prior to the event?	95
None		35	4.2
Beginner		215	26.0
Intermediate		447	54.0
Advanced/Expert		131	15.8

^*Other: Pansexual, Queer, Questioning or unsure, Same gender loving, Other.

Response to Training by Race/Ethnicity

Bivariate associations for race/ethnicity for 861 respondents are shown in Table 2. Most respondents were White/Caucasian (n = 650; 76%), followed by Black/African American (n = 55; 6%) and Hispanic/Latino (n = 43; 5%). Overall perceptions of the training were collapsed into two categories, Yes vs. Neutral/NA or No. Respondents tended to report “Yes” the training met expectations, was relevant, engaging, and increased knowledge; responses ranged from 61% to 98%. However, responses differed significantly by race/ethnicity, each were p < 0.001; those who preferred not to answer the race/ethnicity question rated these items lowest compared to all others. Similar results were shown on ratings for the overall event: 33% of undeclared race/ethnicity respondents rated the event as excellent, whereas almost 66% of Hispanic/Latino respondents rated the event as excellent.

Table 2.

Bivariate associations by race/ethnicity.

Description	n	% Prefer not to answer (n = 39)	% White/ Caucasian (n = 650)	% Black/ African American (n = 55)	% Hispanic/ Latino (n = 43)	% Asian1 (n = 38)	% Other, Multi-Racial (n = 36)	p
Yes, the training…
Met my expectations	860	69.2	92.2	87.3	90.7	94.6	88.9	< 0.001
Seemed relevant to me	859	76.9	96.8	90.9	97.7	97.4	94.4	< 0.001
Engaged me	858	71.8	93.8	85.5	95.3	94.7	94.4	< 0.001
Increased my knowledge	856	61.5	91.2	83.6	86.0	94.6	83.3	< 0.001
Please rate the following aspects of the event:
Overall	836							< 0.001
Excellent		33.3	59.1	64.2	65.9	48.6	42.9
Good		38.5	34.2	28.3	19.5	45.9	42.9
Poor, Fair, or N/A		28.2	6.7	7.5	14.6	5.4	14.3
Which types of resources would you find helpful; all that apply	859
Faculty/staff development for teaching cultural awareness		18.3	21.5	23.2	23.7	24.3	25.5	0.062
Patient-case related resources		15.1	15.5	12.3	19.1	19.6	16.0	0.042
Electronic/Online learning modules		22.6	19.4	20.3	18.3	19.6	21.3	0.962
Links to resources from other		18.3	19.7	16.7	18.3	16.8	13.8	0.268
On-campus workshops		11.8	19.3	22.5	16.0	18.7	17.0	0.096
No additional resources		33.3	12.3	12.7	14.0	2.7	16.7	0.006
Demographics
What is your age?	828							0.004
18 – 34 years		42.1	41.3	22.2	45.2	64.9	37.1
35 years +		57.9	58.7	77.8	54.8	35.1	62.9
What is your gender identity?	745							0.024
Male		11.1	23.5	21.4	19.4	23.5	36.4
Female		77.8	73.6	78.6	80.6	76.5	51.5
Other		11.1	2.9	0.0	0.0	0.0	12.1
Best describes your sexual orientation	818							< 0.001
Heterosexual		15.4	86.5	81.6	78.9	75.8	82.9
All other		2.6	10.6	10.2	13.2	24.2	17.1
Prefer not to answer		82.1	2.9	8.2	7.9	0.0	0.0
Served in any branch of the United States military	850							< 0.001
Yes		7.7	3.9	1.8	7.0	0.0	5.6
No		51.3	95.6	98.2	90.7	100.0	94.4
Prefer not to answer		41.0	0.5	0.0	2.3	0.0	0.0
Do you currently have a diagnosed disability	845							< 0.001
Yes		0.0	3.6	3.6	4.7	2.8	2.8
No		48.7	94.0	94.5	95.3	97.2	97.2
Prefer not to answer		51.3	2.4	1.8	0.0	0.0	0.0
Best describes your PRIMARY affiliation	834							0.217
University Medical Center employee		70.6	70.4	75.5	60.5	59.5	69.4
Student		8.8	14.9	3.8	20.9	16.2	8.3
Hospital employee or other		20.6	14.7	20.8	18.6	24.3	22.2
If you are a student
What is your academic program?	153							0.030
School of Nursing		0.0	3.3	20.0	11.1	0.0	0.0
School of Medicine		50.0	21.7	20.0	11.1	20.0	80.0
School of Health Professions		25.0	67.5	40.0	66.7	50.0	20.0
Graduate Studies		25.0	7.5	20.0	11.1	30.0	0.0
Other		11.1	2.9	0.0	0.0	0.0	12.1
International student	182	0.0	2.8	0.0	50.0	33.3	0.0	< 0.001

¹Asian, American Indian/Alaskan Native, and Native Hawaiian/other Pacific Islander

Responses by Prior Knowledge

Prior knowledge was categorized as none/beginner (n = 250; 30%), intermediate (n = 447; 54%), or advanced/expert (n = 131; 16%; Table 3). Bivariate analysis showed only one significant association between prior knowledge and the question “Did the training increase your knowledge?”. Those who reported they had an advanced/expert knowledge on the topic were less likely to report the training increased their knowledge 71% vs. 94% for none/ beginner and 91% for intermediate. However, because all percentages were high, the training appeared to have value regardless of prior knowledge.

Table 3.

Bivariate associations by prior knowledge.

Description	What was your knowledge of the topic prior to the event?
	n	% None/Beginner (n = 250)	% Intermediate (n = 447)	% Advanced/Expert (n = 131)	p
Yes, the training…
Met my expectations	827	90.4	91.7	86.3	0.176
Seemed relevant	826	96.0	96.6	90.8	0.016
Engaged me	825	93.5	92.8	87.8	0.109
Increased my knowledge	823	94.4	90.6	71.3	< 0.001
Please rate the following aspects of the event:
Overall	803				0.110
Excellent		59.7	56.3	56.3
Good		34.0	35.7	29.7
Poor, Fair, or N/A		6.3	8.0	14.1
Which types of resources would you find helpful; all that apply	826
Faculty/staff development for teaching cultural awareness		54.4	61.0	56.9	0.224
Patient-case related resources		37.2	42.2	52.3	0.018
Electronic/Online learning modules		54.8	53.4	50.0	0.672
Links to resources from other		52.8	50.0	53.8	0.652
On-campus workshops		51.6	51.5	44.6	0.353
No additional resources		13.6	11.9	14.6	0.638
Demographics
What is your age?	796				0.988
18 – 34 years		41.9	41.6	42.4
35 years +		58.1	58.4	57.6
What is your racial or ethnic identification	818				0.031
Prefer not to answer		3.2	4.3	6.9
White/Caucasian		83.9	72.0	68.5
Black/African American		2.4	8.2	8.5
Hispanic/Latino		3.6	5.9	5.4
Asian, American Indian/Alaskan Native, and Native Hawaiian/other Pacific Islander		4.0	4.8	4.6
Other, multi-racial		2.8	4.8	6.2
What is your gender identity?	725				0.221
Male		21.5	23.6	25.0
Female		76.3	74.1	69.0
Other		2.3	2.3	6.0
Which of the following best describes your sexual orientation?	792				0.481
Heterosexual		85.7	81.5	79.4
All other		8.0	11.7	13.5
Prefer not to answer		6.3	6.8	7.1
Have you ever served in any branch of the Unites States military?	823				0.693
Yes		4.4	2.9	5.3
No		92.8	94.8	92.4
Prefer not to answer		2.8	2.3	2.3
Do you currently have a diagnosed disability	820				0.807
Yes		2.4	3.9	4.6
No		93.6	92.1	90.8
Prefer not to answer		4.0	4.1	4.6
Which of the following best describes your PRIMARY affiliation?	815				0.578
University Medical Center employee		67.3	68.8	75.0
Student		16.3	14.5	10.9
Hospital employee or other		16.3	16.7	14.1
If you are a student
What is your academic program?	152				0.178
School of Nursing		3.9	2.5	9.5
School of Medicine		17.6	23.8	28.6
School of Health Professions		74.5	58.8	57.1
Graduate Studies		3.9	15.0	4.8
International student	182	8.5	6.0	0.0	0.351

Responses by Overall Rating for the Training Event

Overall rating for the training was categorized as Poor, Fair, N/A (n = 74; 8%), Good (n = 310; 35%), or Excellent (n = 507, 57%; Table 4). Compared to those who rated the event as excellent, the poor raters were least likely to report the training increased knowledge (96% vs. 38%, respectively; p < 0.001). Several demographic factors differed significantly by overall rating: race/ethnicity, sexual orientation, military service, and disability. Those who identified as White/Caucasian tended to give the training the highest overall rating, as did heterosexuals. Additionally, those who did not serve in the military and who did not have a disability were more likely to rate the training as excellent. Throughout the analysis, one pattern seemed clear: those respondents who self-identified as White/Caucasian heterosexuals with intermediate knowledge of the topic tended to rate the trainings more favorably than any other group.

Table 4.

Bivariate associations by overall rating of the training.

Description	n	% Poor, Fair, N/A (n = 74)	% Good (n = 310)	% Excellent (n = 507)	p
Yes, the training…
Met my expectations	890	54.1	86.8	98.2	< 0.001
Seemed relevant to me	889	64.9	94.8	99.4	< 0.001
Engaged me	888	55.4	89.0	99.0	< 0.001
Increased my knowledge	886	37.8	87.7	96.4	< 0.001
Which types of resources would you find helpful; all that apply	887
Faculty/staff development for teaching cultural awareness		45.8	50.5	64.2	< 0.001
Patient-case related resources		43.1	35.0	43.5	0.050
Electronic/Online learning modules		48.6	45.0	55.9	0.009
Links to resources from other		45.8	44.0	53.8	0.021
On-campus workshops		43.1	43.7	54.7	0.005
No additional resources		23.6	17.5	9.5	< 0.001
Demographics
What is your age?	811				0.044
18–34 years		55.6	40.8	39.1
35 years +		44.4	59.2	60.9
What is your racial or ethnic identification	836				< 0.001
White/Caucasian		60.0	75.5	77.7
Hispanic/Latino		8.6	2.8	5.6
Other, multi-racial		7.1	5.2	3.1
Black/ African American		5.7	5.2	7.1
Asian, American Indian/Alaskan Native, and Native Hawaiian/other Pacific Islander		2.9	5.9	3.8
Prefer not to answer		15.7	5.2	2.7
What is your gender identity?	727				0.822
Male		28.3	21.5	23.9
Female		67.9	75.7	73.3
Other		3.8	2.8	2.8
Which of the following best describes your sexual orientation?	800				< 0.001
Heterosexual		63.8	81.0	84.8
All other		17.4	10.4	10.8
Prefer not to answer		18.8	8.6	4.3
Have you ever served in any branch of the Unites States military?	832				< 0.001
Yes		1.4	5.2	3.6
No		87.0	92.7	95.2
Prefer not to answer		11.6	2.1	1.3
Do you currently have a diagnosed disability	827				< 0.001
Yes		4.3	2.5	4.0
No		81.2	92.9	93.3
Prefer not to answer		14.5	4.6	2.7
Which of the following best describes your PRIMARY affiliation?	818				0.613
University Medical Center employee		67.7	68.6	70.9
Student		10.8	14.5	14.5
Hospital employee or other		21.5	17.0	14.7
If you are a student
What is your academic program?	151				0.151
School of Nursing		12.5	0.0	5.4
School of Medicine		37.5	32.0	17.2
School of Health Professions		50.0	60.0	65.6
Graduate Studies		0.0	8.0	11.8
International student	179	0.0	4.8	4.7	0.801

Thematic Analysis

Responses to the open-ended questions yielded insights into elements of the training that our campus community enjoyed as well as potential areas for improvement. Two questions were analyzed and formed five total themes from responses. Answers to the question, “What did you like best about the training?”, were categorized into three main themes: 1) appreciation for interactivity and multi-modal communication, 2) the power of storytelling, and 3) the learning environment. Answers to the question, “In which ways are you planning to use what you learned today?”, were categorized into two themes: 1) improve general awareness and 2) settings knowledge could be applied.

Question 1: What Did You Like Best about the Training?

Interactivity and Multiple Modes of Communication

An overwhelming majority of the participants valued the interactivity of the trainings. Words such as, “interactive”, “engaging”, and “practical” were used often to describe the training. Participants highlighted the “discussion-based” teaching and the multiple modes of delivery such as videos, pictures, hands-on activities, and the use of personal and institutional examples as having a positive impact on their understanding of a complex topic and satisfaction with the training. One participant shared:

“I enjoyed how the facilitators asked open-ended questions and allowed silence to let participants think through answers. I liked that they had guiding PowerPoints and visuals, but did not simply read off the screen. There was a lot of discussion.”

The Power of Storytelling

Several participants specifically warned against making unconscious bias trainings didactic with comments such as, “I don’t think you should make this a PowerPoint training that people speed through”. Participants consistently commended the use of small and large group discussion during the trainings and the “interaction of everyone at the table and willingness to share”. Participants seemed to value sharing personal stories over the presentation of behavioral and neurophysiological research studies, calling attention to the crucial role that storytelling had on their positive experience of the session. They commented on how, “stories shared painted vivid pictures”, and “storytelling made the whole session very relatable”. The team approach to training allowed facilitators to bring in multiple personal and institutional experiences. Participants noted, “the honesty the presenters had in sharing their biases”. They “liked that there was more than one person giving the presentation. It helped give a few different perspectives”, and said, “the real sharing of the presenters encouraged me to be more vocal”.

Importance of the Learning Environment

Another theme generated from the response analysis was the importance of the learning environment, including the “perceived safety” of the space and avoidance of “shaming”. In their accounts, participants spoke about the “warmth” and “balanced” nature of the environment and how it facilitated meaningful reflections. One participant commented: “The staff created an environment where discussion could take place with everyone’s opinion being equally respected”. Another shared that facilitators were “very personable and engaging, creating a friendly atmosphere”, while a third said “It felt like a safe space, so I felt comfortable participating”.

Comments around “not-shaming” and “normalization of bias” were common, with participants noting that the environment was a “non-threatening environment” that “didn’t call out any particular group or blame anyone”. Participants felt they were “not being put on the spot” and had the “freedom to respond to questions”. They also appreciated that facilitators, instead of shaming, explicitly highlighted “the understanding that it’s okay and natural to have biases, but we need to recognize them and understand their impacts”.

Question 2: In Which Ways Are You Planning to Use What You Learned Today?

Improve General Awareness

Despite positive reviews of the training, participants’ plans to utilize their learning were vague. Some participants answered with general statements such as, “treat everyone the way I want to be treated” or “during interaction with everyone I meet”. Most respondents reflected on the knowledge and skills learned and indicated a commitment to “making myself aware of my bias” and to improving relationships and interpersonal communication, particularly with those belonging to a minoritized group. Some discussed how the training was “very applicable as I work with multiple ethnicities” or how they would use the knowledge when “engaging with others from different backgrounds”.

Settings Knowledge Could Be Applied

Some participants connected the content of the training to their role at the institution, identifying a general intention to use the knowledge acquired to improve recruitment, teaching, and/or patient care. One participant shared they could “use the information obtained from the course with my Doctor of Nursing Practice proposal, interaction with colleagues, and patients I care for”. Others added they would be “thinking about what biases I have when interacting and thinking about students” or “in aspects of teaching/grading and in applicant admissions”. A handful of respondents provided more concrete examples or a plan of action as they shared their plans to apply learning. One specifically indicated a commitment to watching for “biased language in what I write”, while others intended to pay more attention to certain indicators that could signal a need for intervention. For example, during interviews, if an applicant is described as a bad fit, it “means I should take an inward look and explore why I feel that way toward having a person on my team. First quick assumptions aren’t necessarily true.” Similarly, a faculty member noted the training would be “very helpful when interviewing/recruiting residents. If I’m feeling bias or uncomfortable with a candidate, I can stop and ask myself why, plus try to filter out the bias.”

DISCUSSION

This study described the results of a satisfaction survey administered after UB training at a Midwestern medical center. These insights will be used to craft our next steps in terms of UB awareness and education efforts. Results indicated that participants were overwhelmingly positive in their evaluation of the training, with over half rating the sessions as excellent. A large majority of participants rated the sessions as meeting their expectations, indicating it was relevant, engaging, and increased their knowledge. Participants who identified as White, heterosexual, and without a disability tended to give the highest ratings. Participants who identified as belonging to a minoritized group reported finding the content relevant and engaging, however, they had lower satisfaction in terms of the training meeting their expectations and increasing their knowledge. These differences are unsurprising given that White and/or heterosexual participants may be sheltered from experiences of bias while minoritized participants often experience bias in their daily lives in the form of microaggressions.26

Echoing criticisms of UB training,20 the thematic analysis revealed a lack of alignment between the outcomes of the UB trainings (i.e., awareness and improvement of interpersonal communications) and the hopes that our institution had for this effort (i.e., addressing systemic bias and racism). As evidenced by the rapid proliferation of UB trainings, the intensified political unrest of the last three years led many academic institutions like ours to embrace such programs as the silver bullet to achieve equity. As institutions continue to invest time and resources on DEI efforts, their design, the evidence behind their effectiveness, and their limitations must be considered.

Limitations

This study had limitations worth noting. First, the low response rate to the questionnaire, uneven distribution between male and female responders, and an amount of missing data per each question were noted. These might indicate a responder bias. Second, it was unknown how much of the knowledge gained during training will be retained. Conducting a follow-up survey six months after training may help determine knowledge retention. Finally, a small proportion of respondents appeared to be uncomfortable reporting on some key aspects of the questionnaire, selecting the option “prefer not to answer” on such things as age, race/ethnicity, sexual orientation, military service, and diagnosed disability. These are all factors that may be impacted by UB, and most are listed as protected classes from discrimination. Perhaps future training could do more to make all persons more comfortable, such as developing stronger methods for completely anonymous surveys.

There were significant strengths of the project. First, all trainers were certified to facilitate a two-hour UB training event. The certifying organization also provided standardized training material (slides and a participant guide). This ensured that all participants received similar information by trained specialists; thus, events were consistent across all sessions. Second, we had a diverse group of trainers regarding age, race, ethnicity, role, gender, and sex. This may have helped the audience better relate to facilitators and experience safety as evidenced by their positive comments.

Implications for Practice

The results of this study provided several areas of consideration for those who are engaged in DEI education efforts. We propose the following recommendations:

1. Multipronged, interdisciplinary initiatives that are sustained over a long period of time are more likely to render results than annual compliance trainings or one-time trainings. The term “training” seems to suggest a focus on competency development,27 which in the case of UB training might have led adopters to believe that it was indeed a pragmatic solution to elimination of bias. If the institution’s goal is to address systemic racism/isms, then UB training must be situated as part of a larger conversation about systemic oppression, its origins, and how it continues to permeate our society. Academic medical centers may benefit from interdisciplinary partnerships with history, sociology, anthropology, and education experts who are positioned better to provide such offerings.

2. Contextualized programming. While an overall approach to programming may benefit from interdisciplinary partnerships, the need for institutions to develop internal DEI experts was also apparent. Although participants in our UB trainings appreciated the facilitators’ sharing of specific institutional examples of bias, ultimately the analysis also indicated that the training failed to empower participants to develop clear plans of action. Institutions should engage internal experts to develop programming that creates space for attendees to reflect on inequities within context-specific problems, policies, and procedures, as well as potential solutions.

3. Scaffolded professional learning opportunities. The analysis revealed that a one-size-fits all training failed to meet the expectations of underrepresented participants and of those with advanced knowledge on the subject. Some participants self-reported preparedness for a deeper dive into these topics; indeed, they may have come to the training with higher expectations for the depth of content. Providing a continuum of educational experiences could alleviate this satisfaction gap and address the needs of a variety of learners.

4. Adopt a multi-modal, interactive approach with emphasis on storytelling. A strong conclusion from our analysis was that participants appreciated variety in facilitation including use of audiovisual materials, small/large group discussion, and individual reflection. Particularly noteworthy was the value conferred to storytelling by both facilitators and participants over, for instance, presentation of behavioral and neurophysiological research studies. As noted in previous literature,28 storytelling assisted in building rapport, credibility, and trust. It provided a space for those in our audience who identified as White and cisgender to listen and practice perspective-taking, and for folks from minoritized backgrounds to share and feel heard.

CONCLUSIONS

Systemic racism/isms and bias are complex problems to solve. UB training might be a piece of the solution; however, it will not end institutional racism/isms. DEI initiatives should be planned, designed, and evaluated with the same level of rigor and expertise that we demand in patient care, research, and teaching. The results presented in this paper could challenge institutions to evaluate existing DEI offerings critically, to determine what their campus needs are, and to develop their own educational goals and talent as they strive to become more equitable and inclusive.