Association Between Emergency Medicine Clerkship Diversity Scholarships and Residency Diversity

Abstract

Objective

To improve resident diversity, emergency medicine (EM) residencies across the United States have implemented financial scholarships to attract visiting medical students underrepresented in medicine (URiM). The impact of these scholarships on changes in residency racial and ethnic diversity is currently unknown. In this study, we describe characteristics of these visiting elective scholarships for underrepresented students and evaluate changes in residency racial and ethnic diversity after program implementation.

Methods

From 2018 to 2019, we conducted a cross‐sectional survey of EM residency programs with a visiting clerkship rotation scholarship for medical students URiM. Programs were identified for study inclusion using the Society for Academic Emergency Medicine’s online directory of Visiting Elective Scholarship Programs for Underrepresented Minorities. Program characteristics were analyzed descriptively. Changes in residency racial and ethnic diversity were evaluated using an interrupted time series analysis.

Results

Of 34 programs contacted, 20 responded. While there was some variability in funding sources, scholarship amounts, and application review, most scholarships were similar in implementation practices. Of the 20 program respondents, nine were able to provide complete data on residency race and ethnicity and were included in the time series analysis. After program implementation, the time series analysis showed a significant increase in both underrepresented minority EM residents overall and Black and Latinx EM residents in particular.

Conclusion

Emergency medicine visiting clerkship scholarship programs for medical students URiM vary in funding type and application review but had similar implementation practices. Residency programs increased their racial and ethnic diversity after program implementation. Future investigations are needed to determine specific factors contributing to the successful implementation of scholarship programs.

A physician workforce that resembles the composition of our patient population is linked to improved patient care and can address health care inequities. ¹ , ² , ³ , ⁴ , ⁵ , ⁶ , ⁷ However, there is a disconnect between the racial and ethnic backgrounds of emergency physicians and the diversity of the communities that they serve. While Black and Latinx people account for 13 and 18% of the U.S. population, respectively, they account for only 6.2 and 5.3% of medical school graduates. ⁸ Surveys estimate that 1% to 3.1% of emergency physicians are Black and between 1.5 and 3.9% are Latinx. ⁹ , ¹⁰

Black, Latinx, Native Hawaiian, Pacific Islander, Native American, and Native Alaskan medical students are underrepresented in medicine (URiM). Overall, racial and ethnic diversity of EM residency programs has not significantly changed in the past decade. ¹¹ Increasing racial and ethnic diversity in emergency medicine (EM) has become a goal of multiple professional organizations, including the American College of Emergency Physicians (ACEP), the Society for Academic Emergency Medicine, the Association of American Medical Colleges (AAMC), and the Accreditation Committee for Graduate Medical Education (ACGME). ¹⁰ , ¹² , ¹³ In 2017, ACEP published a call for increased diversity in EM to improve patient care and business practices and enhance organizational medicine. ¹⁰ Central to this is diversifying EM residency programs. In 2019, the ACGME published updated Common Program Requirements for accredited residency programs to implement and document policies and procedures related to the recruitment and retention of URiM trainees. These requirements take effect in the summer of 2020, and residency programs may be subject to citation if they fail to develop a plan to enhance diversity in medicine. ¹⁴

There are many strategies that EM residency programs may employ to enhance their residency’s diversity. They range from holistic application review to “second‐look events,” where URiM applicants are invited back to a residency program or institution after an interview. ¹⁵ , ¹⁶ , ¹⁷ An increasingly popular recruitment strategy is the use of scholarships to mitigate the financial burden associated with away EM clerkship rotations. The costs of medical training, including the cost of applying to residency, can place a disproportionate financial burden on URiM medical students and further exacerbate lack of diversity in medicine. ¹⁸ , ¹⁹ , ²⁰ , ²¹ To apply to EM residency programs, virtually all EM applicants must complete at least two senior‐year EM clerkships and submit a standard letter of evaluation from each EM clerkship. While one of these senior EM clerkships is typically completed at the medical students’ home institution, the away rotation is a senior EM clerkship completed at a separate EM residency program. This can be a significant expense for medical students because they must pay for travel, lodging, and other costs associated with completing this clerkship at a location separate from their medical school.

As a part of recruitment efforts to improve residency racial and ethnic diversity, some EM residency programs have invested in scholarship programs for URiM medical students. Currently, the varying characteristics of these programs and their impact on residency racial and ethnic diversity is not known. In this study, we describe characteristics of current EM clerkship diversity scholarships and evaluate whether scholarships can change residency racial and ethnic diversity.

METHODS

Overview

Emergency medicine residency programs with EM clerkship scholarships were identified through the Society for Academic Emergency Medicine (SAEM) online directory of Visiting Elective Scholarship programs for Underrepresented Minorities. ²² This directory, listing EM away clerkship scholarships for URiM students, was established in 2016. At the time it was one of the most comprehensive and accessible sites that was updated on a rolling basis as residencies submitted new or updated scholarship information. In 2018, a total of 34 programs advertised a clerkship scholarship through this platform. We used this list and residency program websites to obtain contact information for scholarship programs, who were generally clerkship directors and residency program directors. We then contacted them by e‐mail and phone to invite them to participate in the study. This study was approved by the Yale Institutional Review Board.

Survey Instrument

We created a survey instrument using an online Qualtrics (Provo, UT) platform to collect data on diversity scholarship programs and sent it to the identified scholarship representative within each program (Data Supplement S1, available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10547/full). Questions included descriptive data of each program’s diversity scholarship, including scholarship initiation, funding, administration, application process, clerkship characteristics, and implementation. Resident racial composition was collected for up to 3 years before scholarship implementation, the year of implementation, and 3 years following scholarship implementation. The surveys were filled out by residency program directors and clerkship directors. When data were missing or seemed inaccurate, we contacted programs via e‐mail and/or phone interviews in an attempt to obtain missing data and resolve discrepancies. Descriptive data were collected from May 2018 to November 2018. Data on resident race and ethnicity were collected from May 2018 to May 2020 to obtain the most recent data on resident composition.

Time‐series Analysis

Using STATA 14 (Stata Statistical Software: Release 14, 2015, StataCorp LP, College Station, TX) we performed an interrupted time series analysis using the .itsa package. We used single‐group time series analysis to evaluate the changes in the proportion of matched URM residents overall and subsequently specifically examined changes in Black and Latinx residents. We used all programs with time data from 3 years before program onset (Pre Y1–Pre Y3), with time of implementation (Y0) and at least 2 years after visiting clerkship scholarship initiation (Post Y1–Post Y3). We defined significance at an alpha of 0.05 and tests of autocorrelation were performed.

RESULTS

We received survey responses from 20 of the 34 EM programs (59%) we contacted with known diversity scholarships in 2018. There was no significant difference between programs who responded to the survey and those that did not based on region (p = 0.270) or length of training (p = 0.110) using Fisher’s exact tests. All 20 respondents gave descriptive information about their EM clerkship diversity scholarships and we were able to obtain resident racial composition data from nine EM residency programs (26%).

Diversity Clerkship Scholarship Descriptive Data

Programs

Twenty EM training programs gave consent and responded to the survey. One of the respondents (5%) did not indicate the identity of their EM program. Of the identified EM programs, all programs are directly affiliated with a U.S. school of medicine, except one. This one program is an undergraduate medical education community site for a medical school and not a core teaching site for the school of medicine.

EM Clerkship Diversity Scholarship Characteristics

Of the respondents, the earliest scholarship was started in 2006 and the most recent was 2018 (Figure 1). Residency programs were located in regions across the United States, and more than half (58%) were 3‐year residency programs (Table 1). Most programs were initiated outside of the EM department (40% initiated by EM department), often supported by the mission of the university (75%) or initiated by a university dean’s office (42%). Scholarship amounts given to students ranged from $500 to $3,000 (missing data from one program). Most programs awarded $1,500, which was also the average scholarship amount (Figure 2). Five programs (25%) indicated that they provided housing for students in addition to the scholarship funds.

Figure 1.

Year of diversity scholarship initiation.

Table 1.

EM Program Survey Responses Describing their EM Clerkship Diversity Scholarship

Program Characteristics	% (n/responses)
Geography
Western region	32% (6/19)
Northeastern region	26% (5/19)
Central region	26% (5/19)
Southern region	16% (3/19)
Residency length
3‐year residency program	58% (11/19)
4‐year residency program	37% (7/19)
Transitioned from 3‐year to 4‐year residency program	5% (1/19)
Reported having an EM department diversity committee	45% (9/20)
Diversity scholarship origination
Reasons for starting the diversity scholarship*
To increase residency class diversity	85% (17/20)
Supported by the diversity mission of university	75% (15/20)
Initiated by the EM department	40% (8/20)
Pipeline for diverse faculty recruitment	25% (5/20)
Requested by EM residents	10% (2/20)
Diversity scholarship was initially proposed by*
University/dean’s office	42% (8/19)
EM faculty	37% (7/19)
EM residency leadership	21% (4/19)
GME office	16% (3/19)
EM residents	5% (1/19)
Department chair	5% (1/19)
Medical students	0% (0/19)
Scholarship funding sources
Department of EM	32% (6/19)
School of medicine (including dean’s office)	16% (3/19)
GME office	11% (2/19)
Multiple sources	42% (8/19)
Scholarship application*
Require a scholarship application separate from clerkship application	95% (19/20)
Require personal statement or essay of interest	100% (19/19)
Require CV	53% (10/19)
Letter of recommendation	37% (7/19)
Medical school transcript	11% (2/19)
Application review
CDs review the application solely	32% (6/19)
CDs and residency PDs review the application together	37% (7/19)
CD, PDs, and clerkship coordinators review the application together	11% (2/19)
Department chair reviews the application with PDs	5% (1/19)
Residents involved in application review with PDs	15% (3/19)
Diversity committee involved in application review with PDs	15% (3/19)
Institutional diversity office involved in review with PDs	21% (4/19)
Criteria for scholarship provision*
Information from EM clerkship application (VSAS)	75% (15/20)
USMLE step scores	90% (18/20)
Medical school GPA/grades	70% (14/20)
Letters of recommendations	35% (7/20)
Personal essays	95% (19/20)
Applicant CV, extracurriculars heavily weighted	25% (5/20)
Measure of scholarship success*
Scholarship recipient matches as their EM residency program	70% (14/20)
Increase in URiM residents in their EM program	45% (9/20)
Scholarship recipient matches at any EM residency program	30% (6/20)
Positive feedback from medical student	35% (7/20)
Faculty evaluations of medical student	30% (6/20)
Other	25% (5/20)
More URiM medical students rotating in the next years	40% (2/5)
There are no measures used to evaluate scholarship	40% (2/5)

CD = clerkship director; PDs = program directors; URiM = underrepresented in medicine.

^*Sum exceeds total as respondents could select multiple answers/responses.

Figure 2.

Amount of diversity scholarship awarded.

Almost all programs (95%) required a scholarship application separate from the general clerkship application. All required a personal statement, and more than half required a CV (53% [10/19]; Table 1). Components of the application deemed important in selecting awardees were the personal essays, USMLE board scores, medical school grades, and information from the VSAS clerkship application (Table 1). A minority of programs (25%) stated that an applicant’s CV, particularly extracurricular activities, was important in selecting scholarship awardees. Programs reported receiving between two and 25 scholarship applications per year with an average of 7.5 applications per program (median = 6, SD = 4.8). More than half (60%) of programs reported that they kept records of applications received each year. Programs reported that they saw an increasing number of applications over time, but this was not quantified. Programs awarded between none and six scholarships per year, and the number of scholarships awarded per year remained steady per program with minor fluctuations of plus or minus one scholarship awarded from year to year.

Scholarship Implementation

Programs defined URiM medical students that would be eligible to receive the scholarships somewhat differently. The majority (89%, 17/19) specifically included race/ethnicity, and about one‐third (32%, 6/19) reported adhering to the AAMC definition of URiM (i.e., racial and ethnic populations that are underrepresented in the medical profession relative to their numbers in the general population). Approximately half (53%, 10/19) of programs included LGBTQ+ students, one‐quarter of programs included applicants who were first in their family to go to college (26%, 5/19) or economically disadvantaged (26%, 5/19). One program provided scholarships to students who demonstrated experience or interest in diversity, health disparities, or serving underrepresented populations, but did not require identification as URiM. The majority of programs (70%, 14/20) measured scholarship success if their scholarship recipient matched at their residency program (Table 1).

Programs identified several common barriers to scholarship program implementation and continuation. More than half (53%, 10/19) identified funding as a barrier. Other barriers identified by programs included difficulty reaching applicants to apply for their diversity scholarship (37%, 7/19), not having direct control over their scholarship program (21%, 4/19, i.e., run by a dean’s office), and difficulty getting sufficient administrative support (11%, 2/19). Most programs (63%, 12/19) identified departmental and institutional financial and administrative support as the main factor facilitating scholarship program success. Less than half of programs (40%, 8/20) reported offering a second‐look opportunity for URiM applicants separate from the diversity scholarship and their EM clerkship. These second‐look events ranged from social activities, such as dinner with EM faculty and residents and tours of the town, to meetings with faculty and residents and ED shadowing opportunities.

Time Series Analysis

We obtained resident racial and ethnic composition data from 13 programs, of the 20 that responded to the survey. Seven programs that responded to the survey did not submit any resident racial and ethnic data. Of the 13 programs who responded, five programs did not have enough postimplementation data on resident race and ethnicity to be included in the time series analysis. The remaining nine programs had at least 2 years of postimplementation data and were included in the time series analysis. Prior to program implementation, URiM residents represented approximately 13.9% of all residents (95% confidence interval [CI] = 11.5% to 16.3%, p = 0.000). Three years after implementation, there was a significant net increase of 3.6% (95% CI = 1.70% to 5.5%, p = 0.009) URiM residents recruited, resulting in URiM residents representing 17.5% of residents in these nine programs at the end of 3 years (Figure 3). Prior to program implementation, Black residents represented 5.6% (95% CI = 3.8% to 7.34%, p = 0.002) of residents. There was a slight, but significant, 2.1% increase of Black residents (95% CI = 0.829% to 3.4%, p = 0.014) after scholarship implementation (Figure 4). Similarly, Latinx residents represented 8.0% (95% CI = 7.41% to 8.60%, p = 0.0001) of residents prior program implementation. After implementation, there was a 1.5% (95% CI = 1.11% to 2.03%, p = 0.002) increase of Latinx residents (Figure 5).

Figure 3.

Overall recruitment of nine residency programs after the implementation of an EM clerkship diversity scholarship (includes Black, Latinx, Native American/Hawaiian/Alaskan, or Pacific Islander). Intervention time year 4 represents the first year of scholarship implementation. Lag (1) indicates the time until the expected effect has taken place. Here we give the program 1 year until the effect of their intervention (diversity scholarship) is implemented.

Figure 4.

Interrupted time series analysis after the implementation of an EM clerkship diversity scholarship of Black medical students to nine residency programs. Intervention time year 4 represents the first year of scholarship implementation. Lag (1) indicates the time until the expected effect has taken place. Here we give the program 1 year until the effect of their intervention (diversity scholarship) is implemented.

Figure 5.

Interrupted time series analysis after the implementation of an EM clerkship diversity scholarship of Latinx medical students to nine residency programs. Intervention time year 4 represents the first year of scholarship implementation. Lag (1) indicates the time until the expected effect has taken place. Here we give the program 1 year until the effect of their intervention (diversity scholarship) is implemented.

DISCUSSION

Our study found some variation between EM clerkship diversity scholarship programs with regard to funding sources, scholarship amount awarded, and the application review process. Core similarities beyond the intended goals of the scholarships were support from the university; application submission components; and importance of personal essays, medical school grades, and board scores for scholarship provision determination. In a subset of programs, we found an increase in residency racial and ethnic diversity after scholarship implementation. While EM resident diversity has not overall increased and most of the diversity scholarships were started recently, we found positive effects within only a few years of scholarship implementation. Our data provide an early indicator that EM clerkship diversity scholarships can increase in resident racial diversity and are worth the time and resource investment.

Diversity scholarships had many similar characteristics and methods of implementation. While most programs highly valued personal essays in their application review, standardized test scores and grades were also important criteria indicated by a significant majority of programs for awarding these scholarships. In recruiting and selecting URiM applicants, this can be important because there is evidence that these traditional measures of accomplishment, competency, and projected success may be biased against people who are URiM. ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ Thus, the criteria for scholarship provision may potentially detract from the goal of diversifying one’s training program. One‐quarter of programs indicated, however, that extracurricular activities weighed heavily in their decision for scholarship awardees. These programs are trying to take a holistic approach to applicant review and strongly considering other measures in awardee selection, such as leadership and advocacy experience. Interestingly, one‐third of programs have the EM clerkship director as the sole reviewer of the scholarship application. This places the onus on one person to make an unbiased decision and may not include the input and vision of residency program or institutional leadership.

Our data indicated a variety in funding sources for diversity scholarships between programs, as well as frequent mention of funding as a barrier for scholarship program implementation and sustainability. Funding likely depends on the degree of institutional and departmental support, as well as degrees of individual support from leaders within a department and residency program. Programs considering a diversity scholarship may be able to find funding, from different offices within a school of medicine and the hospital graduate medical education administration in addition to one’s own department. It is important to consider, however, that funding from medical schools may not be sustainable, because they may not see the value of scholarships for students outside of their institution.

We found an increased proportion of URiM residents in programs offering a diversity scholarship for their EM clerkship. This finding held when examining specifically for Black and Latinx residents at these residency programs. However, increases in residency diversity may not be fully attributable to the scholarship and may represent the impact of multiple, concurrent, unmeasured institutional strategies to increase racial and ethnic diversity. This includes departmental diversity committees and second‐look diversity recruitment events for prospective residents, both of which were present in about half of respondent programs. Since overall EM resident diversity has not changed over the past decade, we do not think our findings merely reflect national residency demographic changes.

While our study demonstrates a positive effect of diversity scholarships for recruitment of URiM students overall, and Black and Latinx residents in particular, we were not able to examine whether there was an effect for Native American, Native Alaskan, Native Hawaiian, or Pacific Islander groups alone due to very low numbers of residents from these backgrounds entering EM in the United States. Most of the residency programs in our study did not have any residents of these ethnicities. If they did, they had no more than one resident of these ethnic groups at a time. We did allow for programs to report “other” for race and ethnicity, but residents categorized as such was a small proportion of the overall sample.

The next steps in further examining EM clerkship diversity scholarships include identifying program characteristics that contribute to increasing residency racial and ethnic diversity. Future research on URiM scholarship recipients themselves could also further examine factors leading to scholarship success and identify issues that detract from these programs.

LIMITATIONS

This cross‐sectional survey has several limitations. First, results may be impacted by selection bias. We used the most comprehensive list of scholarships available in 2018 for survey recruitment; however, it is possible some programs may have not been listed and therefore were not offered study participation. For comparison, current review of both the SAEM and the EMRA website catalogs for EM clerkship scholarships found 62 distinct current scholarships for the 2020 academic year. ²⁸ Many of these scholarships were not available in 2018 and represents the growing popularity of offering diversity scholarships, so the study sample is not fully reflective of the current number of diversity clerkship programs.

Second, results are based on program self‐report and may be subject to recall and reporting bias including information about scholarship implementation and misreporting or misclassification of resident race and ethnicity. To minimize misclassification of resident ethnicity or incorrect reporting of numbers of residents by programs, we worked with residency program leadership to confirm and clarify any discrepancies to minimize misclassification.

This study is also limited by missing data on race and ethnicity. Through the data collection process and correspondence with programs, we learned that over one‐third significant number of programs did not store full records related to their diversity scholarship program. This included the number of applications they received, rejection rates, applicant characteristics, and also race and ethnicity of their residents over the years. Over one‐third of programs that responded did not provide data on the racial/ethnic composition of their residents. This incomplete data collection limits the power of our statistical analysis and our ability to examine characteristics of successful applicants and rates of scholarship provision for racial/ethnic groups. Finally, some programs were excluded from our time series analysis due to timing of scholarship establishment. Many programs had only recently begun during the study period, so although they collected race and ethnicity data, they only had 1 year of resident composition data after scholarship implementation and could not be included in the analysis.

CONCLUSION

To increase racial and ethnic diversity within their residency programs, many emergency residencies in the United States have implemented emergency medicine clerkship diversity scholarships to attract underrepresented in medicine medical students to rotate and match at their institution by defraying the associated costs and conveying their commitment to diversity. Our study showed that while there was some variability in funding sources and application review, most scholarships were similar in characteristics and implementation practices. After implementation, there was a significant increase in underrepresented‐in‐medicine Black and Latinx emergency medicine residents.

Supporting information

Data Supplement S1. URM EM Clerkship Study.

AEM Education and Training. 2021;5:1–10