Qualitative Analysis of Medical Student Reflections on the Implicit Association Test

Abstract

Introduction:

Health professions educators use the Implicit Association Test (IAT) to raise awareness of implicit bias in learners, often engendering strong emotional reactions. Once an emotional reaction ensues, the gap between learner reaction and strategy identification remains relatively underexplored. To better understand how learners may identify bias mitigation strategies, the authors explored perspectives of medical students during the clinical portion of their training to the experience of taking the IAT, and the resulting feedback.

Methods:

Medical students in Bronx, NY, USA participated in one 90-minute session on implicit bias. The focus of analysis for this study is the post-session narrative assignment inviting them to take the race-based IAT and describe both their reaction to and the implications of their IAT results on their future work as physicians. The authors analyzed 180 randomly selected de-identified essays completed from 2013–2019 using an approach informed by constructivist grounded theory methodology.

Results:

Medical students with clinical experience respond to the IAT through a continuum that includes their reactions to the IAT, acceptance of bias along with a struggle for strategy identification, and identification of a range of strategies to mitigate the impact of bias on clinical care. Results from the IAT invoked deep emotional reactions in students, and facilitated a questioning of previous assumptions, leading to paradigm shifts. An unexpected contrast to these deep and meaningful reflections was that students rarely chose to identify a strategy, and those that did provided strategies that were less nuanced.

Conclusion:

Despite accepting implicit bias in themselves and desiring to provide unbiased care, students struggled to identify bias mitigation strategies, a crucial prerequisite to skill development. Educators should endeavor to expand instruction to bridge the chasm between students’ acceptance of bias and skill development in management of bias to improve the outcomes of their clinical encounters.

INTRODUCTION

Over a decade of research suggests physicians demonstrate implicit bias, consistent with findings from the general population.¹ Implicit bias refers to the unconscious, unintentional mental associations we make about various social groups (e.g. race, gender, sexual orientation, etc.) It is a heuristic similar to other cognitive biases, such as those discussed in diagnostic error.² Implicit bias contributes to health disparities through clinical decision-making and communication by health professionals.^1,3 Therefore, implicit bias continues to be a focus of health professions education.^4–17 One of the most widely used tools in implicit bias research and education is the Implicit Association Test (IAT);^1,18 the IAT is an online latency response test measuring reaction times to matching images and value-laden words.^19,20 Within health professions education, the IAT is generally used as a metric to evaluate the effectiveness of educational interventions, or as a catalyst for reflection and discussion.¹⁸ In the former, educators and investigators are seeking to actually change the score on the IAT (i.e. move it towards neutral). In the latter, educators use the IAT as a tool for instruction, not a definitive measure of bias.¹⁸

Recent meta-analyses and narrative reviews found little efficacy for interventions that focus on neutralizing bias alone (e.g. changing the IAT score).^21,22 Our approach to instruction, therefore, is twofold: 1) enable learners to recognize when bias may be influencing a clinical encounter; and 2) promote management of bias to mitigate its negative influence on the encounter.^7,17,23–25 These are essential components for implicit bias recognition and management (IBRM).²⁴ Instruction in IBRM can be guided by Transformative Learning Theory (TLT),²⁴ which emphasizes how awareness and recognition of implicit bias challenges existing assumptions related to being unbiased individuals, engages learners to create a paradigm shift of existing assumptions, and through instruction in skill development to manage recognized implicit bias, empowers learners to act as change agents.²⁶ Briefly, TLT has four parts, an experience, critical reflection, dialogue, and behavior change.²⁶ In our instruction we use the IAT within the experience component of TLT;^6,7,17,27 the experience should be profound, a “disorienting dilemma,” and facilitate deep reflection on the part of the learner.²⁶ In practice, the IAT is often used as an elicitation tool to provoke reflection similar to TLT’s disorienting dilemma.^18,24

Student responses to the IAT have previously been explored in preclinical students and self-selected students during their clinical training.^5,28 Results suggest individuals who complete the test react on a spectrum from being defensive of their results to accepting of them.^5,28 However, the ability to move beyond reaction to the IAT, having recognized bias, and reach strategy identification, a precursor for skill development in management of implicit bias, remains relatively underexplored in the literature. Research on small numbers of self-selected medical students learning in clinical environments who participated in dedicated teaching activities related to strategy identification has demonstrated a change in strategies from pre-session to post-session.⁴ Proposed strategies of pre-clinical students demonstrated some early strategy identification in students lacking a clinical context.²⁸ In contrast, the perspectives of medical students participating in the clinical portion of their education, who have the benefit of clinical context when learning about bias in compulsory learning activities have not been explored. A deeper understanding of learners’ reactions to their IAT results may provide insights to inform educators’ efforts to design instruction in implicit bias relevant to a wider range of students. A more open and robust exploration of how learners react to the IAT and identify strategies to manage their newfound bias awareness is a significant gap in the literature.

Given the important role of clinical learning for professional identity development,^29–31 students immersed in clinical experiences may represent a unique opportunity to explore learners’ reaction to the IAT and their ability to identify strategies to manage their implicit biases. By better understanding how IAT results can foster bias mitigation during this critical time, we can strengthen the quality of IBRM curricula for medical learners. Therefore, the purpose of this study was to conduct an in-depth exploration of the perspectives of medical students in the clinical phase of their education to the experience of taking the IAT after one multi-faceted instructional session, to inform the development of skills-based curricula in IBRM. We sought to explore their reactions to their IAT results and perspectives on the implications of these results on their future clinical care.

METHODS

Medical students in the clerkship year in Bronx, NY, USA participated in a 90-minute session focused on physician implicit racial bias as a contributor to health disparities. Additional details regarding this session are described elsewhere.³² At our institution, the clerkship year refers to year three of a four-year graduate entry medical program, during which students spend their majority time in clinical practice. There was a pre-session assignment asking students to write a reflective narrative on a clinical encounter that illustrated an issue related to health disparities. For the 2013–2017 cohorts of students, this was their only session in implicit bias instruction within the compulsory curriculum. The 2018 and 2019 cohorts were different in that this session was their second instructional session related to implicit bias, having been exposed to implicit bias instruction early in the first year of medical school. Therefore, the reflection prompts used in the 2018 and 2019 sessions were adapted to build on their previous experience (Figure 1). Given previously reported resistance to the IAT when it was assigned prior to any instruction,⁵ we assigned it after the session. We also modified the session in response to previously reported student perspectives by enhancing safety of the learning environment and remove the stigma from the concept of implicit bias.²³

Figure 1.

Changes to 90-minute instructional session on implicit bias recognition and management for clerkship students at a medical school in Bronx, NY, USA from 2013–2019.

The post-session narrative assignment is the focus of this study. It was graded by the course directors. It was part of the final exam, and it accounted for 8% of their total grade for the course. Students completed a written reflection to the following prompt: “The IAT is an interesting tool to help healthcare providers become more aware of their own unconscious assumptions and beliefs. Take the IAT for race. Discuss your reaction to your results and how those results might have an impact on your work as a physician.” A course administrator provided de-identified essays from 2013–2019. All procedures were approved by the Institutional Review Board of the Albert Einstein College of Medicine. As the narratives were required components of the course and no interaction occurred with students for this research, no informed consent was required. Nonetheless, to respect the rights and dignity of our students, investigators only had access to de-identified essays, and all exemplary quotes were carefully reviewed to ensure they were free of any identifying data. We assigned numerical identifiers to each essay and selected them at random (i.e. every fourth essay). We sought to explore perspectives from across classes, therefore we opted to analyze a randomly selected subset of essays.

Five investigators analyzed the data. CMG is clinician investigator focused on implicit bias educational innovations, and contributed to the session revisions described above. YSN was a fourth-year medical student and offered the perspective of having recently participated in the class. NC, HAD, and JWA are course faculty specifically involved in the development and revision of the session. JS is a clinician investigator in medical education with expertise in both qualitative methods and implicit bias.

Our analysis was informed by constructivist grounded theory (CGT) methodology, a suitable approach for exploring social processes and theory building.³³ We also built upon previous research that utilized constructivism exploring reactions to the IAT.^27,34–36 Constructivism emphasizes that knowledge is actively and iteratively constructed as learners build upon prior knowledge.³⁷ We employed an iterative process for data collection, and analysis. We initially selected thirty essays randomly from the 2013–2016 cohorts. Through an inductive approach four investigators independently conducted line by line coding. The coding team engaged in ongoing discussion and analysis, establishing consensus on key codes and their definitions to be included in a preliminary codebook. They split into two sets of pairs and each pair reviewed thirty distinct essays from the 2013–2016 cohorts. These essays were coded independently and in duplicate. Investigators in each pair met to discuss codes, coding disputes were resolved through consensus among the four investigators. Investigators independently and in duplicate conducted axial coding using a constant comparative method to organize the codes and build theory from the findings.³⁸ The investigative group met to discuss all the codes, their meanings and arrive at conceptual themes. Subsequently, an additional investigator joined the team (YSN) and coded thirty randomly selected essays from each cohort form 2017–2019 with CMG using the same codebook to work towards achieving theoretical sufficiency from analysis of these data.³⁹

RESULTS

Our analysis of 180 written reflections demonstrated that clerkship students respond to the IAT through a continuum that includes their reactions to the IAT, acceptance of bias but paralysis for strategy identification, and identification of a range of strategies to mitigate the impact of bias on clinical care. Three main conceptual themes were identified through the analysis process. In accordance with CGT, Each theme iteratively builds upon and relates to the others, highlighting the tensions between learner reactions, acceptance, and challenges related to strategy identification.

Theme 1: Reacting to the IAT

Deep emotional reactions to the IAT permeated student narratives. Students struggled with their reactions to being biased and the implications of having bias on their developing professional identity. One student stated, “I am not just biased. I am biased against my own race! Just like we read about those physicians whose actions were influenced by their implicit bias, could I be guilty of similar actions? The thought hurting anyone based on an implicit bias is terrifying!” (S03177) While others considered the implication of their bias on their future practice:

“I consider [bias] a form of negligence. I really worry that it will affect my work, as it is a trait I was not aware of. … What if it is not just European Americans I have “automatic preference” for over other races/ethnicities? These are questions that I have not reflected on until this point in time.”

(S12903)

There was a general sense of acceptance of bias in oneself and concern regarding its potential negative influence on their future clinical care.

Students also moved past their initial reactions toward a reframing of their biases. For one student their initial humiliation transformed into “gratitude” for their result. Others described an initial dismissive reaction towards the IAT, followed by reflection on the importance of recognizing and managing implicit biases:

…methodological complaints [of the IAT] aside, I do think it is valuable to have potential bias pointed out. It is sobering to realize that we are not perfect, because as medical students and doctors we hope to provide everyone with the best care possible. Tests like these are a good first step for students and doctors to learn about their own biases.

(S64045)

An African American student who completed an IAT that showed a preference towards their own race described that after their initial emotional reaction, their reflections led them to shift their thinking about the possible negative outcomes of future clinical decisions resulting from their implicit racial preference.

Theme 2: Accepting bias yet struggling to identify strategies

Upon reflecting on their IAT results, students feared that they would make biased clinical decisions in the future; they became aware that the effects of implicit bias can potentially harm patients of any race. Students described “worrying” that their IAT results “could be dangerous.” Others highlighted additional sequelae of unmanaged biases on the physician-patient relationship: “Not only am I worried that these implicit biases will affect my clinical judgement towards patients who do not look like me, but I am also worried that those patients might fear me having those implicit biases.” (S18014) Overall, students described a desire to become excellent physicians providing equitable care for their patients. Learning that they had biases from the IAT was regarded as a blow to their perceived identity of the physician they desired to become. They described being motivated to take steps to be more “mindful” of their implicit biases to “strive to be the best physician I can be for my future patients.” (S92138)

Although most students expressed a desire to mitigate the negative impact of their biases on clinical care, they struggled with advancing strategy identification beyond broad and general statements. These included, “it is something I have to work on;” (S06537) “Physicians need to attempt to standardize their care in order to compensate for implicit bias;” (S72193) and “I would like to do everything that I can to nip this in the bud and I hope that it is not too late to change my subconscious attitude.” (S12903) Others described their strategy as being “extra careful” or making an “extra effort.” Students thought they could “suppress” their biases to treat patients effectively. Other students offered no strategies, but rather sought answers to the questions raised by their assignment. One stated, “What are the concrete steps to dealing with such a bias especially from someone of who is found to be biased against his own race? Does this mean that I have possibly discriminated against people of my own race in the past?” (S03177)

Even in the absence of strategy identification, students remained hopeful, expressing their desire to develop strategies to manage biases through their training. Such strategies would enable them to “vanquish them from getting in the way of delivering equal care to my patients.” (S71391) One participant stated, “I hope that with the right strategy to combat my implicit bias, I could avoid becoming like this [making biased clinical decisions] in my practice.” (S20807) These reflections are emblematic of students’ desire to not have implicit bias influence their clinical care, even when they are unable to name a specific strategy.

Theme 3: Self-Identifying a Range of Strategies

Students who were able to move beyond broad, hopeful statements identified a range of strategies to mitigate the impact of their implicit biases on clinical outcomes. In addition to identifying varying strategies, students described the limitations of any single strategy to address implicit biases in a comprehensive way. Some students identified awareness as a sole strategy to mitigate its influence on their clinical care:

I wonder if my work during rotations has already been impacted by my unconscious views…I believe that self‐awareness is the only solution around these biases … This is something that I will start trying on the wards because I believe it will take practice. My hope is that as I enter residency and continue training, this method of self‐awareness will become second nature to me.

(S04780)

Conversely, narratives highlighted the limitations of relying on awareness as a sole strategy to manage implicit biases:

I have found that I sometimes get anxiety thinking about my biases during interactions with Black patients… I really am not sure whether it is a good thing that I know about my implicit bias so that I can act to correct for it or whether knowing about this bias is causing more harm than good by overloading my mental capacity during interactions with Black patients.”

(S66592)

This reflection highlights negative, unintended consequences of awareness as a sole strategy.

Other students’ strategies moved beyond initial awareness to incorporating reflection in real time, or subsequent to the encounter. They described how a feeling of “surprise” can cue them to implicit bias when clinical encounters turn out differently than they expected. Others considered consciously reflecting on treatment options if the patient were a different race: “Whenever I am faced with a patient who is Black or part of another ethnic minority, ask myself whether I would treat a White patient with the same complaints the exact same way.” (S03985) This strategy raised concerns for some of the additional cognitive load: “Isn’t there something concerning that anytime I treat an African American patient I’ll have to take the mental pause to assess if I’ve treated them any differently?” (S02676)

Some students recognized the limitations of attempting constant awareness of bias and considered a team approach to reflection as a useful strategy:

“The problem may be that when fatigue enters the picture and seeing patients may become routine, that approach [awareness of implicit bias] may go by the wayside. Perhaps through regular meetings and pre-scheduled discussions with colleagues, this strategy could remain effective.”

(S51675)

The concept of a team approach to managing implicit bias was echoed by other students. Some believed partnering with others would help them sustain reflection and work towards improving health equity overall, while others advocated for a team approach organized and facilitated through medical education and hospital organizations. One went further by stating that they would seek residency programs that facilitate team reflection on bias:

“I hope to establish communities of safe friends and co-workers who will allow me to share my mistakes and my biases, and who will share with me. I hope to find a residency program that discusses implicit bias and helps create space for talking about our mistakes.”

(S49781)

An additional approach referenced by students involved enhancing objectivity when caring for patients: “Chart review the patient before meeting them or reading their ethnicity, so long as the situation allows it. This way I can make judgments from the medical facts and start coming up with a way to proceed before allowing any implicit bias to intervene.” (S12699) Others considered reviewing their patient cases and reworking them without considering race/ethnicity: “If I manage these ‘blind cases’ differently from real patients, my implicit bias is clouding my clinical judgement to a degree.” (S17669) Additional strategies to enhance objectivity included the use of “checklists” or “protocols.” However, students also suggested approaching objectification strategies with caution. One student who considered this strategy highlighted a potential pitfall, “While this strategy may work well in principle, I don’t think that this is a practical approach to the problem. In reality, each individual and patient encounter is a unique experience, and patient evaluation and formation of a differential is such a dynamic process.” (S17669) Another caution raised regarding enhanced objectivity reflected “a certain detachment from my own feelings when I’m in the hospital… one benefit of this is that any bias I may have about patients, racial or otherwise, is unlikely to affect my decision making.” (S27878) This same student acknowledges this approach has “made me (unfortunately) less empathic, almost robot-like.” (S27878)

DISCUSSION

Feedback from the IAT evoked deep emotional reactions in our students, and facilitated questioning of previous assumptions, leading to paradigm shifts. An unexpected contrast to these deep and meaningful reflections was that students rarely chose to identify a strategy, and those that did provided strategies that were less nuanced. Our study expands existing knowledge on how health professionals such as physicians, nurses, and residents respond to and process feedback from the IAT;^27,34,36 and although other research has explored emotional reactions to the IAT, to our knowledge, ours is the first to explore reactions to the IAT and perceived impact on clinical care in clerkship students after compulsory instruction. Our findings provide insights on the utility of the IAT as a tool for instruction for students immersed in clinical learning. The students in our study demonstrated a desire to provide excellent, unbiased care to all of their patients, yet there is a wide chasm between their desired professional identity and their ability to strategize to manage their implicit biases after recognizing them. This gulf between the professional identity of providing excellent, equitable care to all patients, and students’ inability to identify specific strategies reveals opportunities with implications for enhancement of curricula in IBRM.

Implications for curriculum development

Consistent with previous research with physicians, nurses and residents,^27,34,36 IAT results led to profound emotional reactions in some of our students, a mark of success for a disorienting dilemma.^24,26,40 These emotional reactions highlight the need for faculty to anticipate positive and negative reactions within emotional and cognitive domains.⁴⁰ Although the strategies identified by students tended to be general, there were also strategies that carried potential pitfalls. Our students highlighted the fallibility of constant awareness, and its potential to increase cognitive load. Constant awareness of and striving to manage biases may produce psychological distress.³⁶ Awareness of implicit bias as a sole strategy can also lead to unintended behavioral consequences such as avoidance of members of social groups against whom they are biased.⁴¹ For example, if a student learns they have an implicit White preference, they might avoid Black patients as a consequence of this information.⁴¹

There is also a risk of educators continuing to set unrealistic expectations for learners to be able to modify their behaviors without providing instruction on how to do so. In previous work with resident learners on implicit bias instruction, they identified the need for guidance on bias management.³⁶ We have reported students’ desires for skill development to address their own bias and advocate for their patients when they perceive bias in witnessed encounters.^23,42 The findings of our current study suggest that specific attention to skill development as a component of bias instruction is needed.

Additional potential shortcomings of suggested strategies exist. Reflection may help future encounters, but often will not mitigate the influence of implicit bias in real time. From the patients’ perspective, real time acknowledgement of perceived bias is a viable strategy to restore the relationship.⁴³ Standardizing care and enhancing objectivity has been proposed as a way to mitigate various cognitive biases,^44,45 but no evidence exists to demonstrate efficacy in reducing the effects of racial implicit bias. In addition, evidence increasingly points to bias within protocols and clinical algorithms that may disproportionately burden racial and ethnic minorities.⁴⁶ Such pitfalls highlight the need for a multi-faceted approach for strategy identification and subsequent skill development in IBRM.

A multi-faceted approach aligns with existing frameworks which advocate for a longitudinal approach to IBRM instruction.^25,47 Although a proposed framework suggests that learners may develop skills leading to adaptation of behaviors (reflecting and acting on known biases) and integration into routine clinical practice (recognize previously unknown biases and act to mitigate), they suggest this requires multiple sessions.⁴⁷ Our findings underscore students’ inability to move to adaptation and integration without the opportunity for skill development and practice. Students’ acceptance of bias and their clear desire to deliver unbiased care during their early clinical experiences provides a prime opportunity for skill development.

Our students highlighted the individual nature of implicit bias, and potential pitfalls of any single strategy, therefore, educators should co-construct instruction with learners to bridge the chasm between acceptance of implicit bias and strategy identification. Once relevant strategies are identified, skill development and practice can ensue through active learning exercises. We have detailed opportunities to incorporate active learning exercises related to IBRM to facilitate transformative learning in the past.²⁴ For example, conducting role-plays in which learners need to manage implicit bias in their own or witnessed encounters may be particularly useful for clerkship students. The unscripted nature of the learner role allows for identification of personally relevant strategies and skill development and practice. This co-construction enables strategies to be not only personally relevant, but it relieves educators of the burden of creating multiple strategies for each learner. It allows learners to collectively identify several strategies and practice many skills, thereby countering the realized pitfalls of a single-strategy approach to management of implicit bias.

A team approach to managing implicit bias is a promising strategy identified by our students. Debriefing on implicit bias in a team or small group setting may counter the erosion of students’ professional ideals and ease the frustration of being aware of bias and feeling isolated in their efforts.^23,32 Research studies to date have not specifically studied team interventions, but implicit bias instruction delivered to an entire group of individuals with the same task yielded positive outcomes.^27,48,49 Both students and faculty in our previous work have expressed a desire to debrief about implicit bias.^23,32 Debriefing about bias in a team setting may allow for attempts to manage bias to be discussed so that critical reflection and guided discourse can take place, continuing the process toward behavior change critical to transformative learning over the course of students’ careers.

Confronting their biased selves provokes tension for medical learners who are developing a professional identity of being a good physician providing excellent care to all. The students in our study did not want to be physicians who provide biased care. Evidence suggests students are unable to identify strategies to avoid becoming the physician they do not want to become.²⁹ Poor faculty role-modeling has a negative effect on professional identity formation and can actually increase racial implicit bias.^50,51 Students are socialized throughout their clinical training and without skills to address implicit bias in themselves and others, may alter their ideals to fit in with the team.⁵² Therefore, in order for efforts for a team approach to IBRM to succeed, faculty and residents will need instruction to effectively participate in these discussions. Educators can design instruction to facilitate effective team debriefs through simulation, faculty development programs, and didactics embedded in clinical experiences.

Limitations

We analyzed student narratives for strategies, fully recognizing our prompt did not specifically ask for strategies. We may have gotten specific examples from more students had we explicitly asked for one or two strategies. It is unlikely, however, that students had nuanced strategies that they chose not to share when they were writing about their concerns about potentially providing biased care. Students were writing for a graded assignment, which may have introduced a component of socially acceptable answers. Another limitation is that this study was done in a single institution, and therefore is limited in its generalizability. The purpose of this qualitative investigation was to conduct an in-depth exploration of a relatively understudied phenomenon, not to provide large-scale generalizable results. Finally, although member checking would have been ideal, four of the authors are either a recent student, or instructors in the course. We ensured our depiction and interpretation of the results resonated with their experiences.

Conclusion

Medical students immersed in the clinical portion of their education accept implicit bias and its potential impact on their future clinical care, however providing biased care goes against their perceived professional identities. Current approaches to instruction in IBRM fall short of strategy identification and skill development to achieve this professional ideal. Educators should endeavor to expand instruction from awareness and reflection to active learning exercises for skill development and practice. In the meantime, we risk losing an entire generation of learners who accept implicit bias, but are unable to bridge the chasm between this acceptance to reach skill development in management of bias to improve the outcomes of their clinical encounters.