Increasing URiM Family Medicine Residents at University of Utah Health

Abstract

Introduction

The Department of Family and Preventive Medicine is home for the University of Utah’s Family Medicine Residency program. Although Utah’s diversity is steadily increasing, the race/ethnic diversity of the program’s family medicine residency does not reflect the state’s general population.

Methods

From 2017 to 2021, the residency instituted several adjustments to recruitment processes, including modification of an existing screening system to better highlight resiliency in overcoming challenging life experiences; promotion of commitment to diversity during interview days; incorporation of increased participation from diverse faculty and residents on interview days; and addition of outreach from the Office of Health, Equity, Diversity, and Inclusion. Underrepresented in medicine (URiM) applicants were the first to be offered interviews in an identical screening score cohort, and were ranked highest in rank lists in cohorts with identical final rank scores.

Results

Over the past five match cycles, Latinx residents have increased from zero to six, and underrepresented Asian residents from zero to two. In the 2021 match cycle, five of 10 incoming residents (50%) are URiM. Overall, URiM residents are now 30%, and residents of color 36%, of a total of 30 residents across all 3 training years. We found that eight URiM interviews were needed for every one URiM match.

Conclusion

Intentional resident recruitment initiatives can transform racial/ethnic diversity in a family medicine residency program in a short amount of time.

Introduction

Black, Latinx, American Indian/Alaska Native, Pacific Islander, and Southeast and refugee Asian populations remain underrepresented in medicine (URiM) at levels far below their representation in the general population.^1–3 The growing diversity of the US population necessitates increased diversity in family medicine, as patients from URiM communities often prefer physicians who share their cultural background.⁴ These patients have improved health outcomes when seen by physicians with similar racial/ethnic backgrounds.^5–7 Furthermore, URiM family physicians are more likely to serve URiM communities⁸ and are twice as likely to practice in the inner-city and rural areas.⁹ Utah and national URiM population percentages are presented in Table 1.

Table 1.

Underrepresented in Medicine Groups as a Percentage of Population

Underrepresented in Medicine Group*	US Population14 (%)	Utah Population15 (%)	Utah Physician Population16 (%)
Hispanic/Latinx/a/o	18.1	14	1.6
American Indian/Alaska Native	1.3	1.5	0.4
Black or African American	13.4	1.4	0.5
Native Hawaiian or Pacific Islander	0.2	1.0	0.5
Southeast Asian	n/a	n/a	n/a

^*Our institution defines underrepresented in medicine as persons from American Indian/Alaska Native, Black or African American, Hispanic or Latinx/a/o, and Native Hawaiian or Pacific Islander, and Southeast Asian backgrounds. This information is not disaggregated for Southeast Asians at this time.

To address the lack of URiM physicians in our state, the University of Utah Family Medicine Residency Program (UUFMRP) set a goal to increase the match rates of URiM applicants into our residency program. From 2010 to 2020, 64.9% of all family medicine graduates have remained in Utah to practice, with 77.8% of UUFMRP graduates remaining here to practice in 2020.¹⁰ We felt a systematic change incorporating multiple modalities would be both sustainable and beneficial in our equity, diversity, and inclusion efforts. ^11,12 We report on a successful iterative improvement process.

Methods

The residency program began to critically review recruitment in 2016 and look for ways to improve our URiM match rate. The original process entailed an applicant screening system based heavily on academic achievement, in which points were allotted from data obtained through the Electronic Residency Application Service (ERAS). Our Physician Assistant Division has recruited a high proportion of URiM students for many years, and they shared successful protocols and strategies with us.¹³ We adopted a more holistic approach to screening individual applicants for interview invitations, followed by enhanced promotion of our commitment to diversity, restructuring of interview days, and improving personalized outreach. This study is included in the umbrella institutional review board exemption for educational research obtained by our department.

Applicant Screening

All applications received are reviewed and assigned points based on a scoring system, with the highest-scoring applicants being invited to interview. We reviewed applications with increased attention to overcoming challenging life situations in 2017 and 2018, but the overall number of URiM applicants interviewed was small, and none matched. We added specific resiliency factor points starting in 2019, with an aim to recognize value in diverse life experiences, and we simultaneously deemphasized academic achievement points. Adding points for resilient life experiences increased the number of URiM applicants who scored in the range for an invitation to interview. We further modified point scoring systems during 2019–2021. For the 2020 and 2021 Matches, we also blinded reviewers to applicant photos. Table 2 compares the original scoring system to subsequent iterations.

Table 2.

Original Screening Tool and the Modified Screening Tools to Incorporate Holistic Assessment Utilizing Resiliency Factors

Factors	Original Screening Tool: 2016 and Prior	Modified Screening Tool		Modified Screening Tool	Modified Screening Tool 2021
		2017–2018	2019	2020
Deans’ letter	5 points (Range 1 to 5)	5 points (Range 1 to 5)	5 points (* Range 1 to 5)	7.5 points (* Range −1 to 7.5)	7.5 points (* Range −1 to 7.5)
Family medicine interest and commitment	10 points (Range 1 to 10)	10 points (Range 1 to 10)	5 points (* Range −1 to 5)	7.5 points (* Range −1 to 7.5)	7.5 points (* Range −1 to 7.5)
USMLE/COMLEX:	10 points (Range 1 to 10)	10 points (Range 1 to 10)	5 points (Range 1 to 5)	n/a (only a passing score required)	N/A (only a passing score required)
Medical school/community engagement and leadership:	5 points (Range 0 to 5)	5 points (Range 0 to 5)	3 points (* Range −1 to 3)	5 points (*Range −1 to 5)	5 points (* Range 1 to 5)
Medical school transcript	3 points yrs 1–2 (* Range 0 to 3) 6 points yrs 3–4(* Range −1.5 to 6) 3 pts FM sub-I (Range 0–3)	3 pts yrs 1–2 (Range 0 to 3) 6 pts yrs 3–4 (*Range −1 to 6) 3 pts FM sub-I (Range 0–3)	7.5 points (* Range −2 to 5)	5 points (*Range −2 to 5)	5 points (*Range −2 to 5)
Resiliency points	None	None, however applications read with increased consideration for overcoming challenging life situations	4 points (Range 0 to 4) (1 point for EACH of the following: Economic disadvantage, OR major health event in self, household, or 1st degree relative while in medical school First generation to attend college First generation US citizen or an immigrant Parent during medical school)	5 points (Range 0 to 5) (1 point for EACH of the following: Economic disadvantage Major health event in self, household, or 1st degree relative while in medical school First generation to attend college First generation US citizen or an immigrant Parent during medical school)	10 points (Range 0 to 10) (2 points for EACH of the factors in preceding column)
Resiliency as percent of total points	0.0%	0.0%	13.6%	16.7%	28.5%
Total possible score	42 points	42 points	29.5 points	30 points	35 points

Abbreviations: USMLE, United States Medical Licensing Examination; COMLEX, Comprehensive Osteopathic Medical Licensing Examination.

^*We assigned negative points for failed classes or rotations, negative summative evaluations, and evidence of interest in only a specialty other than family medicine.

Interview Process and Scoring

Each applicant has interviews with three resident or faculty interviewers. Each of these interviewers ask identical standardized questions from one of three areas assigned to their interview with an applicant (Table 3). Points are given for answers based on a structured and consistent system that has been used in our program for over a decade. The application review screening score and the interview day scores add up to the total score for each applicant, which correlates to position in the final rank list. This scoring process has undergone several revisions from 2017 to 2021. In 2021, the application review score accounted for up to 35 points and the interviews accounted for up to 72 points (107 points total).

Table 3.

Standardized Interview Questions

Career Path and Goals
What important factors and experiences have led you to family medicine?
What are some specific elements of this program that are of interest to you, and that you believe will help prepare you for your future practice and goals?
Communication, Compassion, and Insight
Please discuss a clinical case that you’ve been involved with that has had a significant impact on you.
Please discuss a challenging or stressful experience you managed during your years in medical school.
Teamwork and Conflict Resolution
Please tell me about a time that you have chosen to work as part of a team project or community activity.
Please discuss a time when you were part of a team when things did not go well.

Interview Day Interactions

During the interview day, we purposefully highlighted our mission to support diversity in our residents, faculty, and patient populations, and we increased the interactions of applicants with URiM faculty members and residents. Feedback from current URiM residents showed that URiM applicants who see other URiM applicants at interviews have more confidence in the program’s commitment to diversity, so we tried to schedule multiple URiM applicants on the same interview days.

Diversity Outreach Efforts

In the 2019 Match and the 2021 Match, the Offices of Health Equity, Diversity and Inclusion made individual outreach efforts to URiM applicants. This was not done in the 2020 Match, due to staffing shortages external to the residency. Outreach included contacting individual URiM potential residents who interviewed with us via calls, texts and Zoom meetings. In 2021, outreach calls came from people who were not evaluators in the recruitment process (ie, the Assistant Dean for Health Equity, Diversity and Inclusion [HEDI], as well as the Associate Chair for Equity Diversity and Inclusion in the Department of Family and Preventive Medicine).

Results

Table 4 shows the progression of the diversity of the resident group over the last 5 years. Interviews of URiM applicants have increased 16-fold over the last 5 years. During the same time period, the resident group went from zero to nine URiM residents (six Latinx, two Southeast Asian, and one American Indian/Alaska Native). The UUFMRP’s intentional efforts to focus on equity, diversity and inclusion are associated with increasing numbers of matched URiM applicants from 2016–2021.

Table 4.

Residency Recruitment, 2017–2021

Match Year	2017	2018	2019	2020	2021
Total applicants	517	629	599	607	597
URiM applicants	59 (11%)	84 (13%)	104 (17%)	108 (18%)	126 (21%)
Total candidates interviewed	69	78	84	83	98
URiM candidates interviewed	2 (3%)	6 (8%)	16 (19%)	12 (14%)	33 (34%)
Total candidates ranked	68	77	81	80	92
URiM candidates ranked	2 (3%)	6 (8%)	15 (19%)	12 (15%)	33 (36%)
Total candidates matched	8	10	10	10	10
URiM candidates matched	0 (0%)	0 (0%)	3 (30%)	1 (10%)	5 (50%)
Total Resident first year class size	8	10	10	10	10

Total residents July 1, 2021: 30

Abbreviation: URiM, underrepresented in medicine.

Conclusion

Leadership matters in residency recruitment efforts. Our leaders sought a way to increase URiM resident recruitment with (1) a more comprehensive review of applications, (2) deemphasis of traditional academic achievement markers, (3) inclusion of a resiliency scale, and (4) increasing the relative weight of this scale, summarized in Table 5. Changes in the screening process of potential residents to increase diversity has been attempted successfully in other programs,¹¹ and the increase in value for the resiliency points may have played a role in the increase in URiM interviews for the 2021 cycle. Additional efforts, such as highlighting diversity commitment on our residency recruitment materials, verbally discussing our commitments to support diversity, engaging URiM faculty and residents in resident recruitment, outreach phone calls, and collaborating with the GME office and HEDI’s existing URiM resident recruitment incentives, (paid visiting clerkships for URiM applicants and inclusion day panel discussion with current URiM residents—no evaluators permitted) all helped recruitment successes as well. Two of our Latinx residents credited personal connections with University of Utah URiM faculty, and participation in HEDI incentives as influential in ranking the program highly. Multiple applicants noted that the program’s commitment to diversity was evident. The 2021 entering residents also cited outreach calls from the HEDI office as influential in their decision to rank our program. We also determined that, on average, for every eight URiM candidates interviewed, one will likely match. This ratio can potentially be used to guide interview selection for other programs interested in recruiting more URiM residents.

Table 5.

Changes and Interventions in Recruitment, Screening, and Interview Processes From 2017 to 2021

2017	2018	2019	2020	2021
Set a goal increase the diversity of residents in our program	Resiliency points added to screening tool	Changed screening tool to assign negative points if no evidence of commitment to family medicine, or no medical school/community engagement or leadership	Resiliency points increased on screening tool	Resiliency points increased on screening tool
Applications reviewed with increased attention to diversity and overcoming challenging life situations		Phone call outreach to URiM interviewees from OHEDI*	Changed scoring for USMLE/COMPLEX to a pass/fail requirement only	Set a specific goal to interview and match 40% URiM residents
		Program Director emphasized commitment to diversity during interview day presentation and during individual meetings with all applicants	Increased the presence of faculty and residents from URiM backgrounds at some interview days	Phone call outreach to URiM interviewees from OHEDI resumed
			Application reviewers blinded to applicant photos	External residency website and recruitment/promotional information updated to highlight commitment to diversity
				Faculty and residents from URiM backgrounds present at every interview day

Abbreviations: URiM, underrepresented in medicine; USMLE, United States Medical Licensing Examination; COMLEX, Comprehensive Osteopathic Medical Licensing Examination;

^*OHEDI, Offices of Health, Equity, Diversity, and Inclusion (outreach was halted temporarily in 2020 due to external factors).

Our study has several limitations. Because it is a single-institution observational study, its ability to be generalized is uncertain. It is unclear which intervention has the strongest association with the successful recruitment, or if it is a combination of all factors. Future research should include rigorous analysis of data from URiM residents who matched elsewhere, as well as follow-up interviews with those who did match to determine the most important factors in residency selection.

Additionally, the 2020–2021 recruitment and match cycle were all virtual nationally for the first time. An all-virtual interview process may have positively affected the number of URiM applicants, and the number of invited URiM applicants who chose to accept an interview, given the favorable economics of virtual interviews. Maintaining options for virtual interview days in the future may help URiM applicants interview more widely. Focusing on holistic screening criteria and increasing personal outreach can help bring more URiM applicants into family medicine residency programs. Name blinding will be considered in future interventions. These efforts can make individual family medicine residencies more attractive to URiM applicants and increase available opportunities to ultimately expand the URiM physician workforce.