Abstract
Objectives:
To investigate how the decision to report USMLE Step 1 score as pass/fail will influence future otolaryngology residency application and match processes.
Study Design:
Survey study.
Methods:
An anonymous and voluntary survey, approved by the Otolaryngology Program Directors Organization, was administered to academic faculty members from 4/24/2020-5/19/2020.
Results:
257 surveys were received from department chairs (17.5%), program directors (24.1%), associate program directors (12.5%), and department faculty (45.9%). USMLE Step 1 has been the most heavily weighted metric for offering interviews (44.0%), and it has correlated with residents’ medical knowledge (77.0%) and in-service performance (79.8%) but not with surgical skills (57.6%) or patient care (47.1%). In total, 68.1% disagreed with the decision to make USMLE Step 1 pass/fail. This change is anticipated to lead to an increase in significance of USMLE Step 2 CK (89.1%), core clerkship grades (80.9%), elective rotation at the respective institutions (65.7%), AOA and other awards (64.6%), and letters of recommendation (63.8%). The new scoring is also anticipated to especially benefit students from top-ranked schools (70.8%), increase medical students’ anxiety/uncertainty regarding obtaining interview invites (59.1%), and negatively affect international (51.4%), DO (45.9%) and underrepresented students (36.9%). Indication that USMLE Step 2 CK will significantly increase in weight varied according to department position (p=0.049), geographic region (p=0.047), years of practice (p<0.001), and residency program size (p=0.002).
Conclusion:
Most academic otolaryngologists disagreed with changing USMLE Step 1 scoring to pass/fail and believe that it will increase other objective/subjective metrics’ weight and put certain student populations at a disadvantage.
Keywords: USMLE Step 1, Residency Application, Residency Interviews, Step 2 CK, Pass/Fail, Elective Rotation, Letter of Recommendation
Introduction
For years, the United States Medical Licensing Examination (USMLE) Step 1 score has played a major role in evaluating residency applicants especially for competitive specialties such as otolaryngology. According to the 2018 National Residency Matching Program data, the average USMLE Step 1 score for matched otolaryngology applicants exceeded the average score for matched applicants from all specialties by 15 points (mean 248 vs. 233).1 Although it is not a flawless assessment of an individual’s performance and competency as a future physician, the USMLE Step 1 score may elucidate important objective information such as an applicant’s future performance on in-training2 or board examinations3,4 and medical knowledge.5 After more than two years of active debate, the Federation of State Medical Boards and the National Board of Medical Examiners announced in early 2020 that the USMLE Step 1 score reporting will change from the current 3-digit numeric score to pass/fail.6 This is a major shift for both medical students and residency programs, the full effects of which are yet to be determined in the upcoming years.
Although the decision to change USMLE Step 1 scoring to pass/fail was preceded by careful considerations involving feedback from students, residents, faculties, deans, and state medical board officials,7 the effect may not be uniform for all specialties or medical student populations. Other tests such as the USMLE Step 2 Clinical Knowledge (CK) may gain more importance,7 and many programs may seek other standardized/objective metrics or prefer firsthand experience with candidates.8 It is also argued that this transition may negatively affect certain student populations such as doctor of osteopathic medicine (DO) or international medical graduates, underrepresented or minority applicants, and those from less prestigious institutions.9 Given the previously significant importance of USMLE Step 1 scores in evaluating otolaryngology residency candidates,1,10 investigating the potential implications of this transition to pass/fail scoring from the perspective of residency program faculties is warranted.
Methods
This study was exempted from Institutional Review Board approval due to the anonymous, voluntary, and survey-based nature of the research. An online and secure survey was created via RedCap (Nashville, Tennessee) and approved by the Otolaryngology Program Directors Organization (OPDO) task force. The majority of the questions focused on respondents’ perception of USMLE Step 1, its correlation with certain personal attributes of the residents, its importance and utility when evaluating residency applications, and whether specific student groups will be more affected by the transition to pass/fail scoring system. Additionally, the respondents were queried regarding their departmental position and geographic regions. Options for departmental position included department chair (DC), program director (PD), associate program director (APD), and department faculty (DF). Options for geographic location included South, Northeast, West, and Midwest.
The total number of residents in the respective otolaryngology programs was queried with the following options: ≤10 residents, 11-15 residents, 16-20 residents, and ≥20 residents. For statistical analysis, this was categorized as intermediate-sized programs (≤15 residents) and large programs (>15 residents). Similarly, although participants were given four options for the duration of their departmental positions (<5, 5-9, 10-14, and ≥15 years), this was categorized as <10 years or ≥10 years of experience for statistical analysis. In order to prevent missing values, most questions required an answer (i.e. forced response) before the survey could be submitted. The survey, which was voluntary and anonymous, was distributed to U.S. academic otolaryngologists via the Society of University Otolaryngologists (SUO), Otolaryngology Program Directors Organization (OPDO), and Association of Academic Departments of Otolaryngology (AADO) email listservs. Responses were collected from April 24, 2020 to May 19, 2020. PASW Statistics 18.0 software (SPSS, Inc, an IBM Company, Chicago, Illinois) was utilized for statistical analysis, and a p value <0.05 was considered significant.
Results
A total of 257 surveys were completed by 45 DC (17.5%), 62 PD (24.1%), 32 APD (12.5%), and 118 DF (45.9%). The presumptive response rate for PDs was 48.1% (62 PDs from the 129 U.S. otolaryngology residency programs in the National Resident Matching Program’s 2020 Match Report). These departmental positions were held for <5 years (n=99, 38.5%), 5-9 years (n=55, 21.4%), 10-14 years (n=36, 14.0%), and ≥15 years (n=60, 23.3%). Additionally, this cohort consisted of 65 respondents from Midwest (25.3%), 51 from Northeast (19.8%), 84 from South (32.7%), and 57 from West (22.2%). The otolaryngology residency programs represented by the respondents had a total of ≤10 residents (n=42, 16.3%), 11-15 residents (n=98, 38.1%), 16-20 residents (n=59, 23.0%), or ≥20 residents (n=58, 22.6%). Programs that currently use a numeric USMLE Step 1 cutoff score were represented by 80 (31.1%) respondents with an average cutoff of 230.5 ± 8.8, while 29 respondents (11.3%) were unsure and 148 (57.6%) expressed no existing cutoffs for USMLE Step 1. Amongst PDs, 46 (74.2%) did not agree with the decision to make USMLE Step 1 pass/fail, while 30 (48.4%) had a cutoff value for USMLE Step 1 when considering applicants (mean cutoff= 228.5 ± 8.9). The range of USMLE Step 1 cutoffs by PDs (210-245) was comparable to those reported by DCs (220-245), APDs (220-240), and DF (220-245). PDs from smaller programs (≤15 residents) were more likely to report a USMLE Step 1 cutoff compared to larger programs (68.6% vs. 28.6%, p<0.001). Respondents indicating use of a USMLE Step 1 cutoff in the selection process were more likely to consider USMLE Step 1 scores among the top 2 most heavily weighted metrics when evaluating residents compared to those without a cutoff (72.7% vs. 46.7%, p=0.001), but rates of considering USMLE Step 2 CK to have the greatest or second greatest increase in weight were similar (63.7% vs. 54.7%, p=382). When considering their experience with residents, Table 1 demonstrates how the respondents correlated USMLE Step 1 scores with various resident-specific attributes. Notably, 77.0% and 79.8% of faculty members believe that USMLE Step 1 scores correlated with residents’ medical knowledge and in-service exam performance, respectively, while 57.6% and 47.1% believed that it did not correlate with surgical skills and patient care, respectively.
Table 1.
The correlation of USMLE Step 1 scores with various attributes during residency.
| Metrics | Yes | Neutral | No |
|---|---|---|---|
| Patient care | 37 (14.4%) | 99 (38.5%) | 121 (47.1%) |
| Surgical skills | 15 (5.8%) | 94 (36.6%) | 148 (57.6%) |
| Medical knowledge | 198 (77.0%) | 36 (14.0%) | 23 (8.9%) |
| Professionalism and communication skills | 22 (8.6%) | 98 (38.1%) | 137 (53.3%) |
| In-service exam performance | 205 (79.8%) | 44 (17.1%) | 8 (3.1%) |
A total of 175 respondents (68.1%) disagreed with the recent decision to make USMLE Step 1 pass/fail, while 43 (16.7%) agreed with the decision and 39 (15.2%) were neutral on this matter. The proportion of people that disagreed with this decision was similarly distributed across departmental positions (DC 71.1%, PD 74.2%, APD 68.8%, DF 63.6%, p=0.393). Most (n=156, 60.7%) were unsure whether their programs will require a USMLE Step 2 CK score from applicants after transitioning to Pass/Fail USMLE Step 1 scoring, while 82 respondents (31.9%) believed that this will likely be a requirement in the future. USMLE Step 1 is currently the most or 2nd most heavily weighted metric for offering interviews according to 66.2% of the cohort, followed by letters of recommendation (33.9%) and grades in core clerkships (33.5%) (Table 2). The aggregate cohort indicated that after the transition to pass/fail scoring, USMLE Step 2 CK (58.4%), grades in core clerkships (42.0%), and letters of recommendation (26.0%) will have the highest or 2nd highest increase in weight when evaluating residency applicants (Table 2). Overall, 89.1% and 80.9% of the respondents believed that USMLE Step 2 CK and core clerkship grades will have some degree of increase in significance following the transition, followed by elective rotation at the respective institutions (65.7%), AOA and other awards (64.6%), letters of recommendation (63.8%), and research experience (61.0%) (Table 3).
Table 2.
Questions and answers regarding current evaluation of residency applicants and how the transition to pass/fail USMLE Step 1 will impact future application cycles.
| Currently, what are your program’s Most heavily weighted metric for offering interviews? | |
|---|---|
| USMLE Step 1 | 113 (44.0%) |
| Elective rotation at your institution | 34 (13.2%) |
| Grades in core clerkships | 30 (11.7%) |
| Letters of recommendation | 30 (11.7%) |
| Research experience | 11 (4.3%) |
| Currently, what are your program’s 2nd most heavily weighted metric for offering interviews? | |
| Letters of recommendation | 57 (22.2%) |
| Grades in core clerkships | 56 (21.8%) |
| Research experience | 29 (11.3%) |
| USMLE Step 1 | 21 (8.2%) |
| AOA and other awards | 19 (7.4%) |
| Currently, what are your program’s 3rd most heavily weighted metric for offering interviews? | |
| Letters of recommendation | 51 (19.8%) |
| Research experience | 43 (16.7%) |
| Grades in core clerkships | 31 (12.1%) |
| Elective rotation at your institution | 29 (11.3%) |
| USMLE Step 1 | 17 (6.6%) |
| After transitioning to Pass/Fail USMLE Step 1, what other academic cutoffs do you think your program will likely use for screening applicants? (Multiple answers were allowed) | |
| USMLE Step 2 CK | 129 (50.2%) |
| Number of Honors | 148 (57.6%) |
| AOA status | 130 (50.6%) |
| None | 56 (21.8%) |
| Other Research Letter of recommendation MSPE/Class rank School prestige |
74 (28.8%) 26 (10.1%) 10 (3.9%) 7 (2.7%) 6 (2.3%) |
| After transitioning to Pass/Fail USMLE Step 1, which metric do you think will have the highest increase in weight when considering residency applications? | |
| USMLE Step 2 CK | 120 (46.7%) |
| Elective rotation at your institution | 34 (13.2%) |
| Grades in core clerkships | 29 (11.3%) |
| Letters of recommendation | 23 (8.9%) |
| Reputation of medical school | 18 (7.0%) |
| MSPE | 8 (3.1%) |
| After transitioning to Pass/Fail USMLE Step 1, which metric do you think will have the 2nd greatest increase in weight when considering residency applications? | |
| Grades in core clerkships | 79 (30.7%) |
| Letters of recommendation | 44 (17.1%) |
| USMLE Step 2 CK | 30 (11.7%) |
| Research experience | 24 (9.3%) |
| AOA and other awards | 21 (7.8%) |
| Elective rotation at your institution | 20 (7.8%) |
CK: Clinical knowledge; AOA: Alpha omega alpha; MSPE: Medical student performance evaluation
Table 3.
How the importance of several metrics will change after transitioning to Pass/Fail USMLE Step 1, when considering residency applications.
| Metrics | No Change in Importance | Moderate increase in importance | Significant increase in importance |
|---|---|---|---|
| USMLE Step 2 CK | 28 (10.9%) | 100 (38.9%) | 129 (50.2%) |
| Pre-clinical grades | 137 (53.3%) | 89 (34.6%) | 31 (12.1%) |
| Grades in core clerkships | 49 (19.1%) | 108 (42.0%) | 100 (38.9%) |
| AOA and other awards | 91 (35.4%) | 93 (36.2%) | 73 (28.4%) |
| Research experience | 100 (38.9%) | 97 (37.7%) | 60 (23.3%) |
| Letters of recommendation | 93 (36.2%) | 93 (36.2%) | 71 (27.6%) |
| MSPE | 148 (57.6%) | 80 (31.1%) | 29 (11.3%) |
| Personal statement | 197 (76.7%) | 47 (18.3%) | 13 (5.1%) |
| Extracurricular activities | 190 (73.9%) | 50 (19.5%) | 17 (6.6%) |
| Reputation of the medical school | 138 (53.7%) | 68 (26.5%) | 51 (19.8%) |
| Elective rotation at your institution | 88 (34.2%) | 71 (27.6%) | 98 (38.1%) |
CK: Clinical knowledge; AOA: Alpha omega alpha; MSPE: Medical student performance evaluation
The respondents’ level of agreement with how this transition might affect future medical students and residents is demonstrated in Table 4. Specifically, 70.8% of respondents agreed that this will especially benefit students from top-ranked schools and 59-63% agreed that this will increase medical students’ anxiety/uncertainty regarding the number of interview invitations especially at top-ranked programs. Moreover, 83.6% of respondents agreed that this will lead to most programs using USMLE Step 2 CK score as a substitute, and 49.8% agreed that this will negatively impact future residents’ overall basic science knowledge. Lastly, a total of 91 narrative comments (optional) were received and categorized thematically: 20 from DC (9 positive/favorable, 12 negative/unfavorable), 20 from PD (1 positive/favorable, 19 negative unfavorable), 13 from APD (3 positive/favorable, 10 negative/unfavorable), and 38 from DF (14 positive/favorable, 24 negative/unfavorable).
Table 4.
The respondents’ level of agreement with whether changing Step 1 to Pass/Fail will have the following impacts.
| Metrics | Strongly Agree | Agree | Neutral | Disagree | Strongly Disagree |
|---|---|---|---|---|---|
| A positive impact on incoming residents’ overall clinical knowledge and competency. | 8 (3.1%) | 7 (2.7%) | 78 (30.4%) | 105 (40.9%) | 59 (23.0%) |
| A negative impact on incoming residents’ overall basic science knowledge. | 52 (20.2%) | 76 (29.6%) | 84 (32.7%) | 29 (11.3%) | 16 (6.2%) |
| Lead to most programs using USMLE Step 2 CK scores as substitute. | 79 (30.7%) | 136 (52.9%) | 35 (13.6%) | 7 (2.7%) | 0 (0%) |
| Benefit students from top-ranked schools at the expense of those from lower-ranked schools. | 74 (28.8%) | 108 (42.0%) | 48 (18.7%) | 20 (7.8%) | 6 (2.3%) |
| Benefit students that are underrepresented in medicine. | 20 (7.8%) | 64 (24.9%) | 77 (30.0%) | 61 (23.7%) | 34 (13.2%) |
| Put international medical graduates at a disadvantage for matching. | 49 (19.1%) | 83 (32.3%) | 71 (27.6%) | 47 (18.3%) | 6 (2.3%) |
| Put DO medical students at a disadvantage for matching. | 49 (19.1%) | 69 (26.8%) | 86 (33.5%) | 46 (17.9%) | 6 (2.3%) |
| Increase medical students’ anxiety/uncertainty regarding the number of interview invites. | 54 (21.0%) | 98 (38.1%) | 66 (25.7%) | 34 (13.2%) | 4 (1.6%) |
| Increase medical students’ anxiety/uncertainty regarding obtaining interview invites at top programs. | 61 (23.7%) | 102 (39.7%) | 68 (26.5%) | 21 (8.2%) | 4 (1.6%) |
| Improve mental health and wellness among medical students. | 18 (7.0%) | 49 (19.1%) | 90 (35.0%) | 66 (25.7%) | 34 (13.2%) |
CK: Clinical knowledge; DO: Doctor of osteopathic medicine
The 150 respondents (58.4%) who designated USMLE Step 2 CK having the greatest or 2nd greatest increase in weight after the pass/fail transition had a statistically significant difference in geographic location (Midwest 44.6%, Northeast 56.9%, South 64.3%, and West 66.7%, p=0.047). Faculty departmental positions were also different among those that expressed this opinion (DC 62.2%, PD 62.9%, APD 75.0%, DF 50.0%, p=0.049). People holding their respective positions for <10 years were more likely to express this opinion compared to those with >10 years of experience in their departmental position (68.2% vs. 44.8%, p<0.001). Geographic distribution was similar for respondents that selected “elective rotation at your institution” as the greatest or 2nd greatest factor increasing in weight after the transition (Midwest 24.6%, Northeast 21.6%, South 22.6%, West 15.8%, p=0.673). Compared to those from intermediate-sized residency programs, respondents from large programs were less likely to regard USMLE Step 2 CK having the greatest or 2nd greater increase in weight after the transition (67.1% vs. 47.9%, p=0.002). However, rates of holding the same opinion for elective rotations at their institutions were similar (18.6% vs. 24.8%, p=0.226).
Discussion
This survey study of 257 academic otolaryngology faculty demonstrated that 68% did not agree with the decision of making USMLE Step 1 pass/fail, and it was thought that this could especially hurt certain medical student populations while not positively impacting mental health or knowledge level. According to the responses, this transition will lead to an increase in importance for USMLE Step 2 CK, grades in core clerkships, elective rotations at the respective institutions, and letters of recommendations. Additionally, letters of recommendation, reputation of the applicant’s medical school, the Medical Student Performance Evaluation (MSPE), Alpha Omega Alpha (AOA) status, and research experience could also gain significant importance. Although USMLE Step 1 was not thought to associate with patient care or surgical skills during residency, 80% and 77% of the faculties believed it correlated with in-service exam performance and medical knowledge. The study participants were well-distributed among various faculty positions and geographic regions. Although the level of disagreement with the decision to make USMLE Step 1 pass/fail was similar among different faculty positions (range 64%-74%), DF were less likely to believe USMLE Step 2 CK will have the greatest increase in weight (50%) compared to DC, PD, and APD (range 62%-75%). Respondents from intermediate-sized residency programs were also more likely to think USMLE Step 2 CK will have the greatest increase in weight compared to those from larger residency programs. The state of evaluating residency candidates are constantly evolving especially as this decision to make USMLE Step 1 pass/fail has concurred with COVID-19 which will undoubtedly influence the immediate application cycle.11–13 The presented data provide valuable information for medical students and otolaryngology residency programs, who will navigate this uncharted territory for several years until the true effects are discovered.
The previous 3-digit scoring of USMLE Step 1 may have had unintentional negative consequences such as excessive reliance on one test for screening out applicants or discouraging otherwise competent future residents from pursuing certain careers due to not performing in the top percentiles.14 This may especially be the case in programs with a USMLE Step 1 cutoff, which was true with 48% of the otolaryngology programs represented based on PD responses. It is possible that this change to pass/fail scoring will lead to a lengthier application review process, as programs are unable to use an objective cutoff to immediately reject a certain percentage of applicants. Changing USMLE Step 1 scoring to pass/fail may also negatively affect certain student populations, and it is important to discuss the observed thought that this transition can especially impact DO, international, and underrepresented students. In-line with findings of this study, it was previously suggested changing the USMLE Step 1 reporting system will make an already uphill battle more challenging for international medical graduates.15 In a recent commentary by McDade et al., the authors described the potentially complex effect of this change on underrepresented-in-medicine trainees.16 Specifically, this change can be beneficial by reducing the effect of an inequitable test and attending schools with less focus on tested formative elements, but it can also be harmful by the substitution of USMLE Step 1 with other metrics (e.g. school ranking, extracurricular activities, letters of recommendation) that would make these candidates less competitive.16 Interestingly, compared to previous reports that USMLE Step 1 scores can put underrepresented minority applicants at a disadvantage,17,18 most of our respondents were neutral or in disagreement that this change to pass/fail scoring will benefit otolaryngology students that are underrepresented in medicine.
Furthermore, although it has been suggested that USMLE Step 1 examination can be a source of substantial anxiety for students,19,20 most of the respondents believed that the change to pass/fail scoring will actually increase otolaryngology candidates’ anxiety/uncertainty regarding the number of interview invites and chances of matching at top residency programs. This is possibly because USMLE Step 1 is one of the few objective and standardized evaluation metrics, without which more weight can fall on subjective measures such as interview impressions, MSPE narratives, and letters of recommendations.21 Even objectively speaking, the elimination of USMLE Step 1’s 3-digit score can simply lead to many programs using USMLE Step 2 CK as a substitute,22 which was also reflected by the majority of our respondents. Across all medical specialties, it was recently suggested that 81% and 57% of residency program directors believed USMLE Step 2 CK and the applicant’s medical school will have greater importance in the future.23
It has been previously suggested that USMLE Step 1 scores may correlate with residents’ in-training examination,2,24 surgical board scores,25 fellowship success,26 and more favorable faculty evaluations.3 The otolaryngology faculty included in this study largely believed that USMLE Step 1 has historically correlated with medical knowledge and in-service performance, and about half believed that it did not correlate with patient care or surgical skills. A previous review by Bowe et al. demonstrated that USMLE Step 1 score, AOA status, and research/publications were among the most important factors in evaluating applicants, all of which were increasing in score/proportion during recent years.10 As such, it was not surprising to observe that the majority of this study’s respondents believed that these metrics, alongside USMLE Step 2 CK and letters of recommendations, will increase in importance following the transition to pass/fail score reporting. Although this review did not mention the importance of a sub-internship at the respective institutions, this was an important factor reported by this study, which will likely increase in significance among all U.S. regions. This is an interesting notion especially given the current COVID-19 pandemic, where the majority of student sub-internship elective opportunities were canceled. Finally, in the future, schools might investigate ways to express more transparency regarding applicants’ strengths, professionalism, and workplace-based performance for holistic review by residency programs.27
This study contains several limitations that should be considered when interpreting the results. This was a voluntary survey study, thus people with stronger opinions on the topic matter may have been more likely to participate with resultant selection bias. Additionally, the survey software was unable to prevent duplicate responses from the same individual. Due to the effort necessary to complete the survey, it was felt this would be rare if occurring at all, and any negative impact to the quality of the results would be minimal. Since the respondents’ specific programs were also not tracked, some residency programs may have been over-represented, while other programs may not have been represented at all. Given the reported values could be pooled from multiple faculties from the same institution, some results (such as the existence of a USMLE Step 1 cutoff) are best interpreted with data reported by the PDs, as they represent unique programs and are primarily responsible for applicant screening/selection. Administrating similar surveys to medical students pursuing otolaryngology would have also been valuable, but this was beyond the scope of this study. Another limitation was a lack of information on the respondents’ U.S. News and World Report’s program ranking (to protect anonymity), which could play an important role in how they perceive this change and resulting consequences. Similarly, knowing the average score of the current residents at the respective programs could influence how the faculties perceive the importance of this metric, but this was not evaluated. Despite these limitations, this study presents valuable data regarding academic otolaryngologists’ opinions of the change to USMLE Step 1 scoring and its implications on future residency application cycles.
Conclusion
The majority of otolaryngology faculty do not agree with the recent decision to make USMLE Step 1 pass/fail and believe that it may place certain medical student populations at a disadvantage while not benefiting wellness or knowledge. Following the transition, factors that will most likely increase in importance for application review and interview selection include USMLE Step 2 CK, elective rotation at the respective institutions, and grades in core clerkships. Further research may be beneficial to investigate the implications of this important change within other surgical specialties as well as from the perspective of medical students.
Financial Disclosure:
Mehdi Abouzari is supported by the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant TL1TR001415.
Footnotes
Level of Evidence: N/A
Conflicts of Interest: None
References
- 1.Program NRM. Charting Outcomes in the Match: US Allopathic Seniors. https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2019/10/Charting-Outcomes-in-the-Match-2018_Seniors-1.pdf. Published July 2018. Accessed May 30, 2020.Program NRM. Main residency match
- 2.Fening K, Vander Horst A, Zirwas M. Correlation of USMLE Step 1 scores with performance on dermatology in-training examinations. J Am Acad Dermatol. 2011; 64:102–6. [DOI] [PubMed] [Google Scholar]
- 3.Sutton E, Richardson JD, Ziegler C, Bond J, Burke-Poole M, McMasters KM. Is USMLE Step 1 Score a Valid Predictor of Success in Surgical Residency? Am J Surg. 2014; 208:1029–34. [DOI] [PubMed] [Google Scholar]
- 4.Puscas L, Chang CD, Lee H-J, Diaz R, Miller R. USMLE and Otolaryngology: Predicting Board Performance. Otolaryngol Head Neck Surg. 2017; 156:1130–1135. [DOI] [PubMed] [Google Scholar]
- 5.Cassaro S, Jarman BT, Joshi ARet al. Mid-Year Medical Knowledge Milestones and ABSITE Scores in First-Year Surgery Residents. J Surg Educ. 2020; 77:273–280. [DOI] [PubMed] [Google Scholar]
- 6.Boards FoSM. USMLE Program Announces Upcoming Policy Changes. https://www.newswise.com/articles/usmle-program-announces-upcoming-policy-changes. Published February 12, 2020. Accessed May 30, 2020.
- 7.Chaudhry HJ, Katsufrakis PJ, Tallia AF. The USMLE Step 1 Decision: An Opportunity for Medical Education and Training. JAMA. 2020. [Online Ahead of Print] [DOI] [PubMed] [Google Scholar]
- 8.Humphrey HJ, Woodruff JN. The Pass/Fail Decision for USMLE Step 1—Next Steps. JAMA. 2020; 323:2022–3. [DOI] [PubMed] [Google Scholar]
- 9.Lin GL, Nwora C, Warton L. Pass/Fail Score Reporting for USMLE Step 1: An Opportunity to Redefine the Transition to Residency Together. Acad Med. 2020. [Online Ahead of Print] [DOI] [PubMed] [Google Scholar]
- 10.Bowe SN, Schmalbach CE, Laury AM. The state of the otolaryngology match: a review of applicant trends,“impossible” qualifications, and implications. Otolaryngol Head Neck Surg. 2017; 156:985–90. [DOI] [PubMed] [Google Scholar]
- 11.Harary M, Bergsneider M. Letter: Approaches to Mitigate Impact of COVID-19 Pandemic on Neurosurgical Residency Application Cycle. Neurosurgery. 2020. [Online Ahead of Print] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Rodoni BM, Eyrich NW, Fessell DP. COVID-19 & The Residency Match: The Added Importance of Mentoring. Ann Surg. 2020. [Online Ahead of Print] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Quesada PR, Solis RN, Diaz RC, Kraft SM. Otolaryngology Residency Application during the SARS-CoV-2 (COVID-19) Pandemic. Otolaryngol Head Neck Surg. 2020. [Online Ahead of Print] [DOI] [PubMed] [Google Scholar]
- 14.Prober CG. Grading Changes for USMLE Step 1—A Golden Opportunity to Recalibrate Medical Education Priorities. N Engl J Med. 2020; 382:2385–7. [DOI] [PubMed] [Google Scholar]
- 15.Desai A, Hegde A, Das D. Change in reporting of USMLE Step 1 scores and potential implications for international medical graduates. JAMA. 2020. [Online Ahead of Print] [DOI] [PubMed] [Google Scholar]
- 16.McDade W, Vela MB, Sánchez J. Anticipating the Impact of the USMLE Step 1 Pass/Fail Scoring Decision on Underrepresented-in-Medicine Students. Acad Med. 2020. [Online Ahead of Print] [DOI] [PubMed] [Google Scholar]
- 17.Rubright JD, Jodoin M, Barone MA. Examining demographics, prior academic performance, and United States Medical Licensing Examination scores. Acad Med. 2019; 94:364–70. [DOI] [PubMed] [Google Scholar]
- 18.Edmond MB, Deschenes JL, Eckler M, Wenzel RP. Racial bias in using USMLE Step 1 scores to grant internal medicine residency interviews. Acad Med. 2001; 76:1253–6. [DOI] [PubMed] [Google Scholar]
- 19.Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States Medical Licensing Examination Step 1 scores in residency selection. Acad Med. 2016; 91:12–15. [DOI] [PubMed] [Google Scholar]
- 20.Andolsek KM. One small step for Step 1. Academic Medicine 2019; 94:309–313. [DOI] [PubMed] [Google Scholar]
- 21.Pershing S, Katznelson L. The New USMLE Step 1 Paradigm: An Opportunity to Cultivate Diversity of Excellence. Acad Med. 2020. [Online Ahead of Print] [DOI] [PubMed] [Google Scholar]
- 22.Salari S, Deng F. A Stepping Stone Toward Necessary Change: How the New USMLE Step 1 Scoring System Could Affect the Residency Application Process. Acad Med. 2020. [Online Ahead of Print] [DOI] [PubMed] [Google Scholar]
- 23.Alan TM, Matthew EP, Nishant GK, Brian CD. Objective Measures Needed-Program Directors′ Perspectives on a Pass/Fail USMLE Step 1. N Eng J Med. 2020; 382:2389–92. [DOI] [PubMed] [Google Scholar]
- 24.Aljamal Y, Pakonen J, Martin R, Heller S, McKenzie T, Farley DR. Factors that predict an intern′s first ABSITE score are known by September. J Surg Educ. 2018; 75:e72–7. [DOI] [PubMed] [Google Scholar]
- 25.Nagasawa DT, Beckett JS, Lagman C et al. United States Medical Licensing Examination Step 1 scores directly correlate with American Board of Neurological Surgery scores: a single-institution experience. World Neurosurg. 2017; 98:427–431. [DOI] [PubMed] [Google Scholar]
- 26.Yousem IJ, Liu L, Aygun N, Yousem DM. United States medical licensing examination step 1 and 2 scores predict neuroradiology fellowship success. J Am Coll Radiol. 2016; 13:438–44. [DOI] [PubMed] [Google Scholar]
- 27.Swails JL, Aibana O, Stoll BJ. The conundrum of the United States medical licensing examination score reporting structure. JAMA. 2019; 322:605–606. [DOI] [PubMed] [Google Scholar]