Admissions Is Not Enough: The Racial Achievement Gap in Medical Education

Abstract

The achievement gap is a disparity in academic and standardized test performance that exists between White and underrepresented minority (URM) students that begins as early as preschool and worsens as students progress through the educational system. Medical education is not immune to this inequality. URM medical students are more likely to experience delayed graduation and course failure, even after accounting for science grade point average and Medical College Admission Test performance. Moreover, URM students are more likely to earn lower scores on licensing examinations, which can have a significant impact on their career trajectory, including specialty choice and residency competitiveness. After the release of preliminary recommendations from the Invitational Conference on USMLE Scoring (InCUS) and public commentary on these recommendations, the National Board of Medical Examiners and Federation of State Medical Boards announced that the United States Medical Licensing Examination (USMLE) Step 1 would transition from a 3-digit numeric score to pass/fail scoring. Given that another of InCUS’s recommendations was to “minimize racial demographic differences that exist in USMLE performance,” it is paramount to consider the impact of this scoring change on URM medical students specifically. Holistic admissions are a step in the right direction of acknowledging that URM students often travel a further distance to reach medical school. However, when residency programs emphasize USMLE performance (or any standardized test score) despite persistent test score gaps, medical education contributes to the disproportionate harm URM students face and bolsters segregation across medical specialties. This Perspective provides a brief explanation of the achievement gap, its psychological consequences, and its consequences in medical education; discusses the potential effect of the Step 1 scoring change on URM medical students; and provides a review of strategies to redress this disparity.

In February 2020, the cosponsors of the United States Medical Licensing Examination (USMLE) Step 1—the National Board of Medical Examiners (NBME) and Federation of State Medical Boards (FSMB)—announced that the exam would transition from reporting a numeric score to pass/fail scoring. The NBME and FSMB cited secondary uses of the 3-digit score (i.e., the overemphasis on this score and its use in screening applicants for residency interviews) as one of the reasons for this change.¹ This decision came about a year after the preliminary recommendations from the Invitational Conference on USMLE Scoring (InCUS) were released.² These recommendations, especially the recommendation to consider moving Step 1 to pass/fail scoring, resulted in significant feedback from stakeholders, ranging from current trainees to residency program directors to the Association of American Medical Colleges.^3-6 The recent Step 1 announcement has garnered a similar response; of particular note is the concern that with this change, the overemphasis that was put on Step 1 scores will simply transfer to Step 2 Clinical Knowledge (CK) exam scores.^7-9 However, for all of the commentary on the consideration of pass/fail scoring and its consequences, there has been little discussion on the InCUS recommendation to “minimize racial demographic differences that exist in USMLE performance”² or on the specific impact of this scoring change on underrepresented minority (URM) medical students.

Recently, in a commentary discussing this very issue, McDade and colleagues acknowledged that pass/fail scoring does not solve the problem that leads to USMLE score differences between URM and White medical students.¹⁰ Unless medical education undertakes proactive interventions, future residency selection systems risk reinforcing the same disparities that currently place URM medical students at a particular disadvantage in competitiveness for residency, even with the removal of Step 1 scores as a screening tool. In this Perspective, we provide a brief explanation of these racial demographic differences, also known as the achievement gap, their psychological consequences, and their consequences in medical education; discuss the potential effect of the Step 1 scoring change on URM students; and provide a review of strategies to redress the achievement gap.

A Brief Explanation of the Achievement Gap

The achievement gap is a disparity in academic and standardized test performance that exists between White and URM students that begins as early as preschool and worsens as students progress through the educational system.¹¹ The gap is especially pronounced in mathematics and science.¹² Even though it has been about 65 years since the landmark Brown v Board of Education decision, public schools remain highly segregated across the country, and in some states, school segregation has actually worsened since the 1960s.¹³ The expanding achievement gap, driven in part by persistent residential segregation that impacts school funding, results in fewer academic resources for Black and Latinx children.¹⁴ For example, these funding disparities result in higher student-teacher ratios and fewer science and mathematics college preparatory courses in predominantly Black and Latinx schools.^15-18 The unequal quality of K-12 education then results in lower college admissions test scores and lower likelihood of obtaining an undergraduate degree in science, technology, engineering, or mathematics.¹⁹ In many ways, the achievement gap is a resource gap, but these racial disparities persist even in high-quality, well-resourced schools.²⁰

Psychological Consequences of the Achievement Gap

The achievement gap also carries a psychological impact that can further exacerbate the existing disparity. Stereotype threat, the fear that an individual’s performance will be judged in light of negative stereotypes about their social group (e.g., race or gender category) can actually worsen academic performance.²¹ Thus, as students begin to internalize negative messages that their social group is academically inferior, their performance declines.²² In a 2011 study, Owens and Massey examined associations between URM student academic performance and externalization and internalization psychological pathways among ~4,000 college freshmen.²³ The externalization pathway was defined as URMs believing that White students perceived them as less intelligent, whereas the internalization pathway was defined as URMs themselves believing that they were less intelligent than White students. The study showed significant relationships between academic underperformance and both internalization and externalization of negative stereotypes.

Moreover, the impact of stereotype threat also extends to standardized test performance. In low threat test conditions, participants were told they were completing a “series of puzzles,” and Black undergraduate students performed just as well their White classmates on this cognitive ability test.²⁴ However, in standard and high threat conditions, in which students were told they were completing an IQ test, Black students earned lower scores on the same exam. Another study of female students in science, technology, engineering, or mathematics undergraduate programs in Germany showed that internalization of gender stereotypes negatively impacted students’ self-concept even though they earned good grades in their courses.²⁵ The consequences of stereotype threat on coursework and standardized test performance are far-reaching, and students with intersecting oppressions (e.g., Black women) may be especially at risk.

While both stereotype threat and impostor syndrome relate to negative stereotypes, impostor syndrome is defined as an emergent identity that represents how individuals view themselves.²⁶ In a study on impostorism among third-year medical students, (1) women experienced more severe impostor syndrome than men; (2) imposter syndrome significantly correlated with perceived stress; and (3) impostorism was associated with lower Step 1 scores among males.²⁷ As it relates to URM students’ mental health in secondary and postsecondary education, McClain and colleagues observed that both impostor syndrome and “minority status stress” were negative independent predictors of psychological well-being among Black students at a predominantly White university.²⁶ More research is needed in this area, but these findings suggest that URM medical students may also suffer from additional psychological effects, such as stereotype threat and impostor syndrome, that exacerbate the achievement gap. Considering the high-stakes nature of medical school, in which both URM and non-URM students may experience impostor syndrome, it is important to acknowledge and address how psychosocial factors also contribute to academic performance.

Consequences of the Achievement Gap in Medical Education

Medical education is not immune to the achievement gap. Lucey and Saguil have previously described in detail how structural racism impacts Medical College Admission Test (MCAT) score differences across racial groups, and thus disproportionately disadvantages URM applicants.²⁸ Although some students manage to achieve admission to medical school despite society-level disadvantages, matriculation does not erase the effects of decades of educational inequity. URM students are more likely than their White and Asian classmates to arrive to medical school with lower MCAT scores and science grade point averages.²⁹ Black and Latinx medical students are also more likely to receive lower scores on and/or fail licensing exams compared to their White counterparts. In an analysis performed by the NBME, Black, Hispanic, and Asian students were significantly more likely to receive lower Step 1, Step 2 CK, and Step 3 scores than White students.³⁰ These associations persisted even after accounting for science grade point average, MCAT score, and previous USMLE performance. Further, URM students make up nearly half of all medical students who initially fail the Step 1 exam.³¹

This trend has persisted for decades, as a study of medical students in 1992 showed that URM status independently predicted academic difficulty, including delayed graduation and course failure even after accounting for science grade point average and MCAT score.³² Moreover, URM students are more likely to receive lower clerkship grades and are less likely to be inducted into the Alpha Omega Alpha academic honor society than their non-URM peers.^33,34 Clerkship grades are primarily based on shelf exam scores—which have score gaps that are similar to those seen on the USMLE exams—and narrative evaluations, which have been shown to be influenced by evaluator bias.^34,35 These gaps must be understood in the context of unequal resources throughout primary and secondary education. If, because of the structural consequences of racism, URM students are more likely to start from behind their peers in elementary school and never gain adequate resources to catch up (in addition to facing implicit and explicit bias in subjective evaluations³⁴), then their worse academic performance at subsequent stages of education should not be surprising.

Potential Effect of Step 1 Transition to Pass/Fail on URM Medical Students

While the overall effects of a pass/fail grading system for Step 1 seem promising, it is important to consider that, for URM medical students, this decision may actually have unforeseen negative impacts on their medical education. McDade and colleagues recently authored a commentary with compelling opinions on the potential positive overall outcomes and alluded to some of these potential negative impacts for URM students, such as simply shifting the focus from Step 1 to Step 2 CK scores.¹⁰ As a component of the National Resident Matching Program’s 2018 Program Director Survey, residency program directors were asked to indicate and rate factors they used when selecting applicants to interview.³⁶ Of the 33 factors provided, the USMLE Step 1/Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) Level 1 score was the most cited factor at 94%, with an average importance rating of 4.1 out of 5. Other factors cited by at least 70% of program directors included the following: letters of recommendation in the specialty, Medical Student Performance Evaluations (MSPEs), USMLE Step 2 CK/COMLEX-USA Level 2 score, personal statement, clerkship grades, any failed attempt of a USMLE/COMLEX-USA exam, and class ranking/quartile. As with USMLE scores, URM medical students have been found to be at a persistent disadvantage in most of these categories.^30,31,33-35

While the Step 1 exam was designed to determine if an individual meets the minimum proficiency in the basic sciences needed to obtain medical licensure (likewise, the Step 2 CK exam determines this for the clinical sciences), as noted above, these exam scores are considered to be among the most important factors for residency interview selection.³⁶ Furthermore, 64% of program directors indicated using a target Step 1 score and 40% used a target Step 2 CK score when considering which applicants to invite for interviews.³⁶ Residency programs continue to use score cutoffs, even though the data have long supported that this method is disproportionately biased against women and URM applicants.^37-40 If (because of the upcoming Step 1 changes) the weighting of factors used to determine residency interviews transfers to the other highest cited factors, such as Step 2 CK score and class ranking/quartile, then medical education risks perpetuating the same disparities through a new method.

Because the achievement gap affects multiple aspects of medical education, simply shifting the focus from USMLE scores to these other academic factors will not eliminate disparities in residency applicant screening. URM medical students are more likely to receive lower grades than their White peers in their third-year clerkships. In a study of ~3,000 medical students, White students were twice as likely as URM students to receive the highest attainable grade across every core clerkship.⁴¹ In another study of ~1,100 medical students at the University of Washington, Low and colleagues observed that, even after adjusting for Step 1 scores and gender, both URM and Asian students received lower final clerkship grades than their White counterparts.³⁴ They also found that URM medical students received less favorable MSPE summary words. That is, whereas 27% of all students received “outstanding” reviews, only 3% of URM students were rated “outstanding” on the MSPE. Moreover, in a 2018 analysis of differences in narrative language in student evaluations at 2 medical schools, Rojek and colleagues observed that URM and women students were significantly more likely than non-URM and male students to be described by their personal attributes (e.g., pleasant) rather than by competency-related attributes (e.g., knowledgeable).⁴² The importance of narrative language in student evaluations cannot be overstated, as these evaluations are the basis of clerkship grades and are quoted in the MSPE and recommendation letters, all of which are core components of the residency application.

URM medical students’ struggle does not end upon completion of third-year clerkships, as they typically take the Step 2 CK exam immediately following these core clerkships. The Step 2 CK exam correlates to clinical principles, and some research suggests that it may be an appropriate predictor of residency performance and passing specialty board exams.^43-45 Yet, it is important to remember that standardized tests scores are influenced by environmental and nonacademic factors, such as socioeconomic status, and may not assess clinical performance equally across all populations.^46-48 Analyses from multiple residency programs have illustrated that despite lower Step 1 and Step 2 CK scores, URM interns and residents are just as clinically competent as their non-URM peers.^40,49,50 Even if the Step 2 CK exam is a better predictor of career outcomes than the Step 1 exam, medical educators must be cautious not to assign value to residency applicants in a way that reinforces bias. When test scores are overemphasized despite known persistent disparities, medical education contributes to the disproportionate harm URM students face and bolsters segregation across medical specialties.

Given the history of inequities in science and mathematics education and the current weight of Step 1 on career trajectory, including specialty choice and residency competitiveness, the elimination of numeric scoring is a step in the right direction. However, the factors that contribute to Step 1 score gaps also influence Step 2 CK scores and clerkship grades. Therefore, early interventions for URM students remain vital. The InCUS recommendations and Step 1 announcement demonstrate that there is an urgent need to address racial achievement gaps in undergraduate and graduate medical education. In the next section, we will discuss programs that seek to redress these inequities before the residency application cycle.

Strategies to Address the Achievement Gap

The achievement gap was created and reinforced by decades of structural disparities, and it persists at every socioeconomic level.⁵¹ We do not expect medical educators to bear the burden of repairing the entire society, nor do we assume that all URM students will perform poorly in medical school. However, it is important to understand the context in which a significant proportion of URM students arrive to medical school. Namely, it is the responsibility of medical education institutions to be strategic in addressing one of the largest obstacles that URM medical students are likely to face: standardized test performance.

The creation of interventions for at-risk students, who are much more likely to be URM students, is not new to medical education. Many of these programs, ranging from prematriculation programs to programs to improve test-taking skills, have shown gains in academic performance for these students.^52-64 However, few institutions have implemented a comprehensive overhaul of medical curricula. One institution that has done this is the University of Texas Medical Branch at Galveston, which implemented an integrated preclinical medical curriculum that emphasized problem-based learning, early identification and intervention for potentially at-risk students, and opportunities for deliberate practice through an internal question bank provided by the administration, among other measures.^57,58 These global changes in the curriculum resulted in a 14-point increase in mean Step 1 scores overall, but Black students experienced a 21-point mean score increase and a 94% decrease in Step 1 failures.⁵⁹ Integrative approaches that are targeted and proactive (such as the one used by the University of Texas Medical Branch at Galveston) improve outcomes for all students, but the results for URM students in particular suggest that these types of approaches may be a potential means of closing the achievement gap.

Although there has been extensive research on the efficacy of preclinical interventions,^52-59 studies on the impact of supplemental programs for the clinical curriculum and Step 2 CK exam are lacking. Given that racial gaps persist on both Step 1 and Step 2 CK, the absence of programming after the preclinical curriculum is particularly concerning in light of the Step 1 scoring change. Simply put, if institutions see a need for intervention programs to prepare students for Step 1, that need persists throughout medical education. These interventions must be both comprehensive and longitudinal to fully close the achievement gap for URM students.

Conclusions

The achievement gap between White and URM students exists across all educational areas and levels in the United States and is not limited to standardized test performance. Importantly, medical education is not exempt. The impact of centuries of structural racism on the achievement gap is complex. While holistic admissions are a step in the right direction of acknowledging that URM students often travel a further distance to reach medical school, medical education cannot forget about these barriers once biochemistry lectures begin. A holistic admissions process that is not married to holistic medical education is simply not enough.

Here, we specifically discussed the racial achievement gap and its consequences in medical education. Not only do URM students cope with the psychological challenges often associated with being a URM in higher education, but they also face a climate that places undue emphasis on standardized test scores. And while the NBME and FSMB have decided to no longer report numeric scores for Step 1, the achievement gap will persist without deliberate, structural change. Academic interventions should be proactive and not delayed until URM and at-risk students are already failing. Evaluation measures during clerkships should be standardized, and implicit bias training is imperative. Residency programs should consider the consequences of using any standardized test score as the primary screening tool for extending interview offers. Policymakers should advocate for educational equity in pre-K through postsecondary education.

Admittedly, this area is difficult to address, but it is important that medical education affords all students an equal opportunity to excel. If medical education is to be truly committed to diversifying the medical profession (and all of its specialties), it must develop and implement specific, actionable strategies to close the achievement gap in medical education that begin during the preclinical years and extend throughout the curriculum.