Creating a Diverse Physician Workforce Through Training in Diversity, Bias, and Selection: An Innovative Approach to Preparing Medical Students as Future Academic Leaders

Abstract

Medical students are the future of academic medicine. They will serve as admissions committee members, deans, and program directors with responsibility for selecting future physicians. At the same time, schools and programs are working to diversify the physician workforce to care for diverse patient populations. At UAB Heersink School of Medicine, we developed an elective course, Holistic Review: Creating a Mission-Driven Physician Workforce, to train learners in bias, diversity, and data-informed decision-making. Students applied lessons to the admissions process by participating in application screening and multiple-mini interview rating. Student reflections demonstrated course applicability for careers in medicine and beyond.

Background

Medical students will become the gatekeepers to medical school, residency programs, and faculty positions as admissions committee members and deans, residency program directors, and department chairs. Few students, however, receive formal training in selection processes or in the biases that impact equitable and mission-driven selection. This lack of training jeopardizes the selection process, impeding the recruitment of a workforce poised to meet the needs of diverse patient populations and to reduce medical disparities for underserved and marginalized groups.

The need to diversify the healthcare workforce has been well-documented [1–3]. In fact, medical schools and some residency programs are employing holistic review principles and processes to align selection with program-specific missions and diversity interests, positively impacting the physician workforce across community clinics and large academic healthcare centers. Students educated in diverse learning environments report feeling better equipped to meet the needs of diverse patient populations [4]. Additionally, communication may be improved between patients with healthcare providers from similar backgrounds [5] and perceived bias of healthcare providers leads to less compliance with treatment plans and negatively impacts treatment and health outcomes for underrepresented groups [6].

Since medical students represent the future of leadership in medical education, our objective was to introduce them to a holistic admissions process that supports and enhances diversity through its design. To our knowledge, there have been no previous reports outlining a course with this specific content or learning objectives. Therefore, through a qualitative research lens, we wanted to understand the educational impact this curriculum had on the learners who participated.

Activity

We designed and implemented this elective course in the fall semesters of 2019, 2020, and 2021 to medical students enrolled at the University of Alabama at Birmingham Marnix E. Heersink School of Medicine. The course, Holistic Review: Creating a Mission-Driven Physician Workforce, was based on the work of the AAMC Advancing Holistic Review Initiative [7] and studies supporting the effectiveness of holistic review processes in impacting diversity [8–10]. Course topics included history of medical school admissions, holistic admissions review, theories of meritocracy and social capital, mitigating bias, racism in healthcare, compliance with legal and accreditation standards, evidence-based selection tools, and holistic review in student affairs and GME. Students applied their learning by participating in application screening and by rating participants in multiple mini interviews (MMIs) (Table 1). Course instruction included a combination of didactic and small group discussion. Guest lecturers joined the class virtually to share their expertise from multiple medical schools across the country. Course content remained the same; however, the delivery format shifted from in-person to virtual in 2020 and 2021 due to COVID-19 safety concerns.

Table 1.

Course topics

• History of medical school admissions • Holistic review • Theories of meritocracy and social capital

• Mitigating bias

• Racism in healthcare

• Compliance with legal and accreditation standards • Evidence based selection tools • Application of lessons in application screening and multiple mini interviews (MMIs)

• Holistic review in student affairs and GME

Course instruction in every session included a combination of didactic and small group discussion

After course completion, students wrote reflection papers on their experiences and learning. Students selected two prompts from a list of four possible prompts in 2019 and a list of three possible prompts in 2020 and 2021 (Table 2). One of the prompts in 2019 was eliminated in 2020/2021 due to low selection rate and overlap with other prompts. For the purposes of this study, qualitative responses were only analyzed for the three remaining prompts.

Table 2.

Prompts used for reflection papers

• In what ways did the course confirm your previous understanding of holistic review? In what ways did the course differ from your previous perception(s) of holistic review?

• How might this course relate to your career path? Describe what you learned that will help you in your career?

• What was your favorite part(s) of the course? Describe what you learned. How will you apply your learning?

Learner responses to the prompts were de-identified prior to analysis, assuring a blind coding process. An inductive approach to thematic analysis, as described by Braun and Clarke [11], was applied to the responses to code the data and identify overall themes. Inductive analysis was chosen to allow a broad, data-driven analysis free from the researchers’ existing frame. Initially, all authors familiarized themselves with the data by reading through the responses to each prompt. One author then manually coded the responses to generate the initial codes. Initial codes were then reviewed by all authors and sorted into preliminary themes. Preliminary themes and conflicts of opinion were discussed, refined, and organized across all prompts until consensus was reached to generate the final themes and subthemes.

Results

Forty medical students took the course in Fall 2019, twenty in Fall 2020, and seventeen in Fall 2021. Enrollment numbers were deliberately reduced after 2019 to allow for more individualized learner feedback. All medical students were either in their second or third year of medical school. Each student responded, in detail, to two prompts regarding their previous understanding and lessons learned about holistic review, applying what they learned to their future careers, and/or their favorite parts of the course. Four main themes, and multiple subthemes, were established from the analysis of learner responses to the three prompts (Table 3).

Table 3.

Identified themes and subthemes

Theme 1: Complex, Evidence-Based Subthemes:

• Time and resources intensive

• Data and evidence driven

• Individualized, contextual evaluation of experiences, attributes, and metrics Sample Quotes: • “I assumed the choice to participate and screen applicants based on holistic review was an easy one and didn’t use more resources than the traditional screening model. Now, however, I realize that the holistic review model is time and labor intensive, and requires a large sacrifice on the part of the school and its implementers to make sure appropriate training and personnel are available to do the task. While it is a system that requires a lot of effort to implement, I do believe it provides us with quality results and has the potential to shape the field of medicine as we know it over the next fifty years.” (Learner 4, 2019) • “Another perception I had before this class about holistic review that is now different is the way applicants are evaluated. Coming into the course I thought that due to the complex nature of everyone’s application there were different expectations and standards given to each applicant. I learned this is actually the opposite. In order for holistic review to work there needs to be a strict rubric in which every applicant is assessed against. This allows for standardization across screeners and fairness in the process.” (Learner 6, 2020) • “The course proved that holistic review is really an assessment of each applicant individually, independent of the others, to learn what they can contribute to the success of the class. This process seeks to see and acknowledge each person from where they are in their own journey, and to see how that journey aligns with the mission of UAB.” (Learner 20, 2020) • “I was unaware of the large number of people needed to make holistic review possible and the amount of time that goes into training committee members. I had no idea the amount of data and algorithms that are used in the decision making. I did not know all of the research being done in holistic review and the evidence used to guide practices. I also did not know that holistic review is largely an individualized process for each school. There are guidelines from the AAMC and workshops for institutions, but how they are able to implement it is up to them.” (Learner 1, 2021) • “My previous beliefs about the screening process and how students are invited to interview and are accepted were very different from reality. While I knew that UAB employed a holistic review of applicants, my interpretation of holistic review was very limited. Through screening and trainings on screening, I truly learned just how important an applicant’s story and experiences are. Often an applicant’s background can greatly influence opportunities available to them, academic performance, and many other aspects of their application. While I had always assumed that life challenges that an applicant faced would play a role in the admissions process, it was not until this course that I learned about accounting for distance traveled and how this affects rating.” (Learner 13, 2021)

Theme 2: Mission-Based Subthemes:

• Selection criteria based on mission

• Application of mission-match to career decisions Sample Quotes: • “Learning about holistic review and applying this concept by reading applications has definitely shaped the type of advice that I will be giving to premedical students when I become a doctor. Essentially, holistic review and how it comes into play in the admissions process vary depending on how “experiences” are measured at each school and what each medical school is looking for. Since all medical schools have their own mission statement, they are all looking for applicants that will specifically fit their school. Some schools are focused more on research while others are more focused on primary care. Rather than trying to fit the ideal “pre-medical student mold,” applicants should focus more on getting involved in activities that they are passionate about and focus on what they took away from it. When I become a full-fledged doctor, I hope to be involved in the admissions process and apply the concepts I learned from this course.” (Learner 30, 2019) • “In regard to hiring and screening potential employee applications, this course was helpful in bringing to my attention the role of holistic review in the workplace. I previously did not think of hiring employees as a process in which holistic principles could be applied. I see now the importance of identifying goals/missions for the workplace and evaluating applicants with these markers. It is important to have these goals/missions clearly stated when evaluating applicants because it helps make the process more objective and reduce the effects of people’s inherent biases.” (Learner 37, 2019) • “While this course was focused on holistic review in the medical school admissions process, we talked multiple times about how it could be applied to any sort of selection committee whether that be in residency selection process, or employment outside of medicine. The holistic review of mission base selection can be applied to most aspects of my career goals. My end career goal is to be in small private practice taking care of patients in my community. This course has helped me understand more about how to go about selecting individuals for a position when there are more applicants than positions available. The idea of mission-based selection was a new concept to me. Before this class I thought that generally the best way of selecting people for positions was merit based but I learned that in most cases in medicine that is not the best approach. There is so much more to being in healthcare than knowledge, yes, it is an important part but not the only component. In the future, when I am searching for people to help work with me in my practice, I now have some tools such as defining a mission and creating a standardized a rubric to see how potential co-workers/employees align with that mission to help me with that process. I feel that the holistic approach could also be applied in reverse. Where many programs use mission-based criteria to select applicant for positions I feel that the same approach could be used for applicants to select programs. For example, this class taught me the basics on how to design a holistic approach and when applying to residency programs in the future I could set out criteria for what I want in a residency program to help me decide which program is the best fit for me and my career goals.” (Learner 6, 2020) • “I would also say that, before this course, I did not realize how you could tailor holistic review to your school’s specific needs and goals. I always thought of holistic review as a broad term, but this course showed me how it can be used to look for specific traits or interests that your school values. For instance, we talked about how UAB uses compatibility with its mission points in the admissions process and even had the opportunity to see that evaluation in practice through application screening.” (Learner 8, 2020) • “Lastly, I did not realize the extent to which holistic review allows individual institutions to recognize criteria that benefit their own needs. For instance, UABSOM places an emphasis on recruiting in state residents or people with specific ties to Alabama. Alabama needs this and benefits from this, because we have a shortage of primary care physicians and care in rural locations and need physicians who will be inclined to serve in Alabama. Overall, I was very pleased to learn about the multiple ways that holistic review can not only benefit individuals, but also benefit the needs of greater causes, such as health care to an entire state.” (Learner 13, 2020) • “I also think the holistic review process will be helpful in many other aspects of my life and career in which I have to make decisions based off limited information. For instance, I will be applying to residency programs in a few years. Whole programs will certainly be evaluating me, I as an applicant also play a role in choosing where to go. I will have to decide what my own ‘mission matches’ are that make a program a good fit for me. I think deciding what these missions are for myself beforehand is important. Without these it is tempting to rank places based off where the residents are similar to you, or where you just happened to ‘click’ with a couple of people on your interview day. That is not to say inter-relational components are not important, but keeping a holistic mindset will certainly help me to be more objective in my decisions and ensure that I end up at a place that truly fits my goals and aspirations.” (Learner 6, 2021)

Theme 3: Diversity Subthemes:

• Mitigating bias

• Promoting diversity, equity, and inclusion

• Impact on the learning environment

• Impact on patient care and medicine Sample Quotes:

• “I think regardless of what kind of career in medicine I choose in the future, what I have learned in this course could be applied to a multitude of things. Even if I choose a career in private practice far from academia, I think that the overarching themes of bias, privilege, and holistic approaches to people will serve me far in improving how I interact with others… Throughout this course, I learned a lot about how to approach evaluating an applicant as a whole, and how to place proportionate emphasis on different categories. I also learned more about implicit and explicit bias and how they can affect how we view the people we interact with. Most importantly, this course has taught me how it is crucial to not make quick assumptions about people, and to see applicants as truly a whole person with different perspectives and skills to contribute.” (Learner 35, 2019) • “As a future physician who will one day be tasked with taking care of a diverse set of patients from different backgrounds and walks of life, I feel the training in unconscious bias that we did will prove extremely beneficial to me in my future career. The training made me reflect and [I] realized that within the walls of an academic classroom, it is easy for us to be blinded to and ignore our own biases. We have the choice to interact with who we want, form study groups with who we want, and even hang out with who we want outside of school. It’s an artificial setup in many ways because we often gravitate towards those people we feel are similar to us or that we have more in common with whether that be religion, politics, education, hometown, etc. We essentially can create an environment for ourselves in which we are rarely if ever challenged to confront our own biases or get out of our comfort zone. Now that I am in the hospital and looking to my future career path in medicine, I realize that no matter the specialty I choose this same artificial setting will not be present. I will be in countless situations where the person sitting across from me is different from me whether they be a colleague or patient. I realize that these settings challenge me to get out of my comfort zone and make me a better friend and physician. In our session, I was reminded that it is my responsibility to check in with myself to make sure that the quality of care I am providing to patients is equivalent to what I would want for myself and that when I find myself disagreeing or getting frustrated with a patient or colleague that I need to step back and look at it from their perspective. Most likely, they aren’t operating from the same framework, assumptions, or set of beliefs that I am. I also realized I need to routinely check in with myself to see if there is bias on my part playing a role in my reaction to the situation and work to overcome that bias if present. The knowledge and self-reflection I gained from the unconscious bias for healthcare professionals workshop will prove beneficial to me not only in my career as a physician but also in my personal life as I interact with and support those around me.” (Learner 4, 2019) • “I hope to apply what I have learned in this course in any position of leadership I may have in the future (whether in the field of healthcare or otherwise). Remaining objective when assessing applicants requires one to acknowledge any implicit biases they may have and ensuring they do not affect the final decision. This is critical in the field of healthcare, as a patient may initially appear noncompliant to a physician, but there could be other factors at play that the physician is not considering.” (Learner 7, 2020) • “The main concept that was reinforced in this course that will help me in my career is the fact that each person is more than their application or patient chart. I have found myself forgetting this fact and made assumptions about a patient and their life based on their health. This class has helped me check my assumptions and bias when I see patients. We were taught to think about applicants beyond their academic accomplishments and take into account their background growing up, possible support or lack thereof while applying and other limiting factors to their academics or activities. I think it makes a lot of sense when you break each person down that there are many complex situations and personality traits beyond what is listed on the application. This class helped me check my own bias for applying to medical school because I went to a large university that had strong pre-professional programs, I had a ton of support and advice for my application, advice for my pre-graduate curriculum, strong courses to prepare me for the MCAT and resources to prepare for interview. My experience made me blind to the fact that other applicants might not have gone to a school that offered as much support. Instead of my original assumption that the applicant was lazy and did not put any effort into his or her application there is the simple explanation that the person went to a small school without a pre-professional program, graduated many years ago before deciding to pursue medicine, and many others. This will help me in my career, so I am able to understand and remember that my patients have live a different life than my own. It is very easy to be frustrated with patients for not taking their medications when I know the patient will benefit from them. You easily think, ‘how hard is it to just take a few pills each day’ but there are so many barriers to medication adherence…In my career I will continue to be cognizant of my patient’s social situations and its effect on their health. I will ask more questions to understand my patients beyond their health to serve them best, make a difference in multiple aspects of their lives, and help them have positive experiences in the healthcare system.” (Learner 1, 2021) • “A big thing I can take away from this course is learning how to actively and consciously be aware of any biases I may have had that I wasn't fully aware of before screening applications. This can allow me to be a better overall doctor as well. Bias can definitely impact a patient in the healthcare system so being aware of my own biases and setting them aside will help ensure a more thorough and safe healthcare experience for any patients I may see in the future. Biases like schools an applicant were tied to, or life experiences were things I realized I had to be aware of and set aside so that it didn't impact my judgment on who the applicant is at a fundamental level. This course has definitely made an impact on me and will help me going forward throughout my career to be a better physician. I think the biggest takeaway for me from the course is that I constantly need to keep myself aware of any potential sub-conscious biases that may come up throughout my career when seeing patients. I have already started thinking about this when seeing simulated patients whereas before I wasn't really keeping that thought running through my mind. Also, it will serve to help me have a positive impact on an application committee if I am to get that opportunity one day.” (Learner 4, 2021)

Theme 4: Application of Holistic Review Principles Beyond Medical School Admissions Subthemes:

• Guidance for preparing own applications for residencies and jobs

• In other selection processes: hiring future staff, residency programs

• Patient-care

• Work teams • Personal life Sample Quotes: • “I think that the content of this course can apply to many fields, not just medicine. In almost any line of work, people have to select the next generation of workers and employees. The way that selection is made has been the same for many years. People look at resumes, have a short interview, and pick the person with the most impressive qualifications. However, if we want our workforce to be more inclusive and diverse, people who are doing the selection for jobs and must work to actively mold the workforce… I hope to one day be in a position where I am able to influence how the next generation of doctors is chosen. I would love to serve on an admissions committee for medicals school or on a resident selection committee. I know I will carry many principles that I learned in this course and use them when the time comes.” (Learner 11, 2019) • “Reviewing applications has also helped me on the wards. In order to be an effective application reader, it is important to get a clear, full picture of each applicant. The personal statement, extracurricular activities, and letters of recommendation contribute important aspects of the applicant, so processing this information is essential to understanding the applicant and the contribution he or she will make to our medical school and as a future physician. A similar skill is needed on the clerkships. Patients come in with extensive medical histories, complex social histories, and many personal concerns. To get a good idea of each patient, it is important to look at these individual aspects together to better treat and cater to the patient as a whole. This course and the opportunity to review applications has encouraged me to look at patients holistically. I will continue to practice this throughout the rest of my clerkships and throughout my medical career.” (Learner 24, 2019) • “In addition, there are many teams where I may be tasked with making personnel decisions in my career as a physician. Currently, I am a part of the Primary Care Track which could require me to hire a complete staff if I were to eventually open up my own practice. The principles of holistic review could be a useful means by which to select the nurses and other support staff members whom I will rely on in my daily practice. Even if I do not end up in primary care and pursue one of my other interests such as emergency medicine, I may be asked to provide input for residency interviews or other hiring practices. Essentially, the principles of holistic review can provide value in any sort of selection process. If you have a good understanding of holistic review and what you are looking for in an applicant, you can build a criteria by which to assess applicants and find ones that best fit your needs while evaluating for effectiveness as you go. If you do not see the results you are looking for from your selection process, it is best to go back and reassess your criteria. Holistic review is a useful tool for the selection process that I think will benefit me throughout my career whether I am on the side of being selected or being the selector.” (Learner 8, 2020) • “I believe that this holistic review course will serve me well in multiple dimensions of my career path. I think the origins of the patient-doctor relationship may be approached similarly to sitting down with an applicant's AMCAS application. First, you must review all of the medical information, medical history, prior surgeries and hospitalizations, medications and notes from other healthcare providers to prepare for your first interaction with the patient (analogous to screening an applicant's experiences, personal statement, grades, and letters of recommendation before the in-person interview). There's only so much information that may be extracted from the patient's chart; the initial patient interview and HPI are crucial to comprehending their healthcare journey and how the patient response to medical advice (analogous to MMI and one-on-one interview bridging any gaps found in the AMCAS application). Only when you have reviewed the patient's chart, attained a thorough patient history, and listened to the patient's story will you be ready to address a patient’s problems in a holistic manner. This is incredibly important in patient care because humans are more than just numbers and lab values on a chart. The foundation of their health problems may be buried deeply beneath superficial medical information; therefore it is a physician's role to encompass a multifaceted approach in take into account the words and perspective of the patient to not only explain healthy behavior, but also provide the highest level of care to someone in need.” (Learner 3, 2021)

Learner responses in each dataset were blinded and randomly assigned an identifying number for qualitative analysis. The quotes listed above are labeled with the assigned learner number and dataset year

Many students had little prior knowledge of the real-life application of holistic review and came to the course harboring doubts regarding its rigor and standardization in a formal review process. Students were surprised to learn about the complexity of the holistic review process and reported a greater appreciation for the highly structured and data-informed way in which it was conducted. After participation, students reported a greater understanding for how holistic review evaluates applicants individually, exploring the context in which students completed their experiences. Participation in the course brought awareness to the importance of selection criteria, in addition to grades and test scores, and highlighted how factors such as “distance traveled” are considered. Distance-traveled was defined as the educational and socio-economic barriers overcome prior to medical school. Students also reported a greater appreciation for the time and resources required to holistically review medical school applications. Prior to the course, many viewed the application process as a metrics-based competition, but now appreciate how a holistic approach is individualized to each applicant and allows for the selection of a class that embodies institutional missions and goals.

Students were intrigued by the value placed on mission-based matching for medical school admission using a holistic review process. Many respondents had not previously considered this topic and were surprised to learn how the holistic review process can be tailored to meet a school’s specific needs and goals, while upholding their values. Furthermore, students now understand the importance of considering an institution’s mission as they move forward with their own career decisions.

Prior to participating in the course, students understood holistic review to be about embodying diversity in the selection process and assessing applicants on multiple factors in addition to standard academic metrics. However, students reported defining diversity more narrowly prior to participation, with a gained appreciation for how lived experiences also drive diversity among applicants. After participation, students believe increased diversity in the medical school class can positively contribute to the learning environment as well as the profession of medicine. The course curriculum, and the opportunity to review medical school applications, caused many students to reflect on their own unconscious biases when faced with individuals who challenged their views of ideal applicants. Students reported value in gaining this personal insight and appreciate how lessons learned about mitigating biases will help them in caring for patients, interacting with colleagues in the workplace, and their personal lives.

After participation in the course, students reported a greater understanding of how holistic review principles can be applied outside of medical school admissions. In addition to preparing their own applications for residency and jobs, they now have a greater appreciation for how to evaluate others in a more holistic way. Students recognized this skillset is not limited to the review of medical students or residents, but also in decisions for hiring future staff and interacting with a diverse patient population.

Discussion

Diversity in healthcare is imperative for better healthcare, research, and patient outcomes. There are many barriers to creating a diverse physician workforce. This innovative course provided formal training for medical students in holistic admissions review, mitigating bias, and mission-based selection to increase the knowledge and skills of future gatekeepers to careers in medicine.

While the course was designed to prepare medical students for possible future careers as academic leaders, student reflections demonstrated that lessons may be applicable to other aspects of their lives and careers. In addition to diversifying the physician workforce, course components, such as mitigating unconscious bias and promoting diversity in medicine, encouraged students to apply lessons learned, collectively, for improving patient care, the workplace, and their personal lives. Students felt the lessons prepared them for more strategically applying to residency programs, to better mentor pre-medical students and, perhaps more importantly, to be more competent clinicians for diverse patient populations.

While the course confirmed previous assumptions about holistic review involving more than academic metrics, the students reflected on how the course helped them to understand the complex, data-informed, and mission-driven aspects of the holistic review process. Many also expanded their perspective of diversity to include life experiences beyond demographic data. Their favorite parts of the course involved hands-on experiences in selection using mission-aligned evaluation tools. In a healthcare environment with increasing diversity and persistent gaps in healthcare for underserved and marginalized groups, this course prepared medical students to understand the importance of a diverse workforce, to mitigate biases in selection, the workplace, and patient care, as well as to use mission-driven, data-informed processes in selection. This study may be expanded in the future to include tracking student careers and participant feedback on applicability at various stages of their careers.

Data Availability

All data is available upon request.

Declarations

Conflict of Interest

The authors declare no competing interests.