Systems-Level Reforms to the US Resident Selection Process: A Scoping Review

Ryley K Zastrow; Jesse Burk-Rafel; Daniel A London

doi:10.4300/JGME-D-20-01381.1

. 2021 Jun 14;13(3):355–370. doi: 10.4300/JGME-D-20-01381.1

Systems-Level Reforms to the US Resident Selection Process: A Scoping Review

Ryley K Zastrow ¹, Jesse Burk-Rafel ², Daniel A London ^3,^✉

PMCID: PMC8207920 PMID: 34178261

Abstract

Background

Calls to reform the US resident selection process are growing, given increasing competition and inefficiencies of the current system. Though numerous reforms have been proposed, they have not been comprehensively cataloged.

Objective

This scoping review was conducted to characterize and categorize literature proposing systems-level reforms to the resident selection process.

Methods

Following Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines, searches of Embase, MEDLINE, Scopus, and Web of Science databases were performed for references published from January 2005 to February 2020. Articles were included if they proposed reforms that were applicable or generalizable to all applicants, medical schools, or residency programs. An inductive approach to qualitative content analysis was used to generate codes and higher-order categories.

Results

Of 10 407 unique references screened, 116 met our inclusion criteria. Qualitative analysis generated 34 codes that were grouped into 14 categories according to the broad stages of resident selection: application submission, application review, interviews, and the Match. The most commonly proposed reforms were implementation of an application cap (n = 28), creation of a standardized program database (n = 21), utilization of standardized letters of evaluation (n = 20), and pre-interview screening (n = 13).

Conclusions

This scoping review collated and categorized proposed reforms to the resident selection process, developing a common language and framework to facilitate national conversations and change.

Introduction

Calls for substantive reforms to the US resident selection process are growing, given increasing competition and inefficiencies of the current system.¹ Over the last decade, applicants have doubled the number of applications they submit.²^–⁴ Inundated by applications, programs are increasingly reliant on filters such as United States Medical Licensing Examination (USMLE) Step 1 scores for screening, despite its bias against minorities and poor prediction of residency performance.⁵^–⁸ With Step 1 transitioning to pass/fail in 2022, programs may utilize Step 2 Clinical Knowledge (CK) for screening in the absence of other reforms.⁹

Even if programs could conduct a holistic review of all applications, limited standardized data are available for review: clerkship grading distributions vary between institutions,¹⁰ Medical Student Performance Evaluations (MSPEs) lack standardized objective measures,¹¹^,¹² and narrative letters of recommendation are subjective and omit applicant shortcomings.¹³ Once interview offers are extended, a “first-come, first-served” frenzy occurs among applicants to secure an interview,⁴^,¹⁴ often with competitive applicants hoarding interviews only to cancel last minute.¹⁴^,¹⁵ Couples, osteopathic (DO) graduates, and international medical graduates (IMG) face additional barriers.¹⁶

Numerous reforms have been proposed to improve the efficiency, equity, and integrity of the resident selection process. However, many proposals are published in specialty-specific journals, and a comprehensive compilation of all reforms is currently lacking.⁴^,¹⁷^,¹⁸ Therefore, this scoping review was conducted to characterize systems-level reforms to the resident selection process. We aim to provide educational leaders with a clear framework and consistent language to facilitate national discussions.

Methods

The protocol was drafted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) and prospectively registered with the Open Science Framework on February 20, 2020.¹⁹

Search Strategy

Designed by a health science librarian, comprehensive searches of the Embase, MEDLINE, Scopus, and Web of Science databases were conducted in February 2020 for articles published from January 2005 to February 2020 (provided as online supplementary data). In an attempt to capture all reforms presented in editorials, commentaries, and letters, the table of contents of the following undergraduate and graduate medical education journals were manually searched: Academic Medicine, Journal of Graduate Medical Education, Advances in Health Sciences Education, BMC Medical Education, Teaching and Learning in Medicine, and Medical Teacher. Furthermore, bibliographies of included articles were manually searched for additional relevant articles. All records were imported into Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia), and duplicates were removed.

Inclusion and Exclusion Criteria

English language articles proposing systems-level reforms to the US residency application, selection, and match process were included if applicable to all applicants, medical schools, or residency programs, regardless of specialty. Articles were excluded if no reform was proposed, a previously proposed reform was mentioned without explicit endorsement, the reform referenced a resident selection process outside the United States, or the reform was not generalizable beyond the individual program (eg, program-specific resident selection protocols, hiring independent contractors to assess program culture). Articles proposing reforms to the fellowship match process were also excluded.

Screening and Full-Text Review

References were independently screened by 2 authors (R.Z., D.L., or J.BR.) for inclusion based on their title and abstract. Articles then underwent full-text review by 2 reviewers (R.Z., D.L.). A third reviewer (J.BR.) reconciled discordant cases at screening and full-text review.

Data Extraction and Synthesis

Two reviewers (R.Z., D.L.) extracted data in duplicate. Extracted data included the type of reform, implementation strategies, cited advantages and disadvantages of the proposed reform, and the specialty targeted by the reform as suggested by the journal in which the article was published or authors' affiliations. Articles were classified as “not specialty specific” if authors were from multiple specialties or if the proposed reform was explicitly applicable to multiple specialties.

Qualitative content analysis using an inductive approach with grounded theory was performed to categorize reforms.²⁰ First, 2 reviewers independently read a subset of included articles and created preliminary codes. Reviewers then reconciled their findings, refined preliminary codes, and generated a codebook. The process was repeated for subsequent articles in multiple groupings, creating new codes as needed. Finally, similar codes were categorized into higher-order categories and themes. ATLAS.ti 8 (ATLAS.ti Scientific Software Development, Berlin, Germany) was used to aid in higher-level category creation and assessment of code co-occurrences.

Results

Of the 10 407 unique references identified, 225 proceeded to full-text review and 116 were included in the scoping review (Figure). Though many references were not specialty-specific (n = 38), otolaryngology (n = 22), orthopaedic surgery (n = 16), general surgery (n = 8), emergency medicine (n = 7), and plastic surgery (n = 5) were the specialties most frequently proposing reforms (Figure provided as online supplementary data). Over the study period, there was a steady increase in the number of articles proposing reforms each year (Figure provided as online supplementary data).

Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Flow Diagram of Search Strategy

Our inductive approach generated 34 codes that were grouped into 14 categories and then organized according to broad stages of the resident selection process: application submission, application review, interviews, and the Match (Table 1). Additionally, the pros and cons of each reform, as reported by the reviewed articles, were compiled and summarized (Table 2).

Table 1.

Proposed Systems-Level Reforms to the US Resident Selection Process From 116 Articles Meeting Inclusion Criteria (January 2005–February 2020)

Category	Code (with article count and references)	Definition
Reforms to the Application Submission Process
Application cap	Application cap (n = 28)⁴^,¹⁵^,¹⁸^,²¹^–⁴⁵	Limit the number of applications submitted per applicant
Signaling program preference	Supplemental applications (n = 11)⁴^,¹⁸^,²⁶^–²⁸^,⁴⁰^,⁴⁶^–⁴⁹	Require program-specific supplemental applications (eg, essays)
Signaling program preference	Signaling system (n = 5)²⁴^,⁴⁴^,⁵⁰^–⁵²	Enable applicants to designate programs as “preferred”
Standardized program information	Standardized program database (n = 21)⁴^,⁵^,¹⁷^,¹⁸^,²⁴^,²⁷^–²⁹^,³³^,⁴¹^–⁴³^,⁵⁰^,⁵³^–⁶⁰	Create a centralized database with standardized program information
Standardized program information	Establish residency brand (n = 6)¹³^,¹⁷^,³³^,⁴⁸^,⁶³^,⁶⁴	Define and advertise program culture and applicant traits likely to fit well
Reforms to the Application Review Process
Holistic review	Holistic review (n = 12)¹⁸^,⁴⁰^,⁴¹^,⁶⁰^,⁶⁵^–⁷²	Evaluate academics in context of personal/professional experiences
Medical school grading, examinations, and metrics	Standardized grading (n = 2)¹⁰^,⁷⁵	Standardize preclinical and clinical grading across all medical schools
	New medical school assessments (n = 4)⁴^,¹⁸^,⁶⁶^,⁶⁷	Create assessments for residency selection (eg, simulations, EPAs)
	New medical school metrics (n = 1)⁵	Establish new metrics to capture applicant medical school experiences
Residency evaluations and metrics	Residency selection examinations (n = 1)¹⁷	Create a general residency selection examination separate from USMLE examinations
Residency evaluations and metrics	Specialty-specific metrics (n = 3)⁶⁵^,⁶⁸^,⁷⁷	Create new examinations/metrics to capture specialty-specific knowledge/skills
Noncognitive assessments	Psychological assessments (n = 13)¹³^,¹⁷^,⁶⁴^,⁶⁷^,⁷²^,⁸³^–⁹⁰	Use noncognitive assessments (eg, situational judgement tests, grit)
Noncognitive assessments	Personality assessments (n = 10)¹⁷^,⁷²^,⁷⁸^–⁸⁵	Utilize personality assessments to evaluate applicant traits
Medical Student Performance Evaluation (MSPE)	Require objective data in MSPE (n = 11)⁴^,¹¹^,¹²^,¹⁸^,³¹^,³⁹^,⁵⁹^,⁹¹^–⁹⁵	Require objective data in MSPE (eg, class rank, grades)
	Standardize MSPE structure (n = 8)¹¹^,¹²^,⁵⁹^,⁶⁰^,⁷⁰^,⁹³^–⁹⁵	Standardize structure, content, and language of MSPE
	Third-party review of MSPE (n = 1)⁵	Third-party MSPE review to assess compliance with AAMC guidelines
Standardized letters of evaluation (SLOE)	Utilize SLOEs (n = 20)¹¹^,¹³^,¹⁷^,²⁷^–³¹^,⁴⁸^,⁶⁰^,⁹⁰^,⁹⁶^–¹⁰⁷	Implement SLOEs to uniformly summarize applicant performance
	National SLOE norms (n = 3)²⁷^,⁹⁰^,¹⁰⁶	Create nationally normed rubrics and database of SLOE writers
	SLOE guidelines (n = 1)¹⁰⁷	Publish guidelines for letter writers to aid in SLOE preparation
	Visual letters of recommendation (n = 1)¹⁰⁸	Generate visual LORs from SLOE (eg, word clouds)
Pre-interview screening	Pre-interview screening (n = 13)²¹^,²⁶^,²⁹^,⁵⁷^,⁶⁴^,⁶⁶^,⁶⁸^,⁶⁹^,⁷⁶^,⁸⁸^,¹⁰⁹^,¹¹¹^,¹¹²	Conduct pre-interview screening by preliminary video/phone interviews
Interview allocation and scheduling	Standardize interview offer dates (n = 7)⁴^,¹³^,²²^,⁵³^,¹¹³^,¹¹⁴	Standardize dates of interview offers with an acceptance window
	Online interview scheduling (n = 1)⁵⁷	Utilize an online interview scheduling system with real-time availability
	Interview match (n = 1)¹¹⁵	Require applicants and programs to submit rank lists for interviews
	Interview lottery (n = 2)³¹^,¹¹⁴	Create a lottery whereby interview dates or slots are randomly assigned
	Interview cap (n = 6)²¹^,²⁹^,⁷⁶^,¹¹³^,¹¹⁶^,¹¹⁷	Limit the number of interviews applicants may attend
	Standardize interview dates (n = 5)²⁹^,⁵⁹^,⁷⁶^,¹¹⁶^,¹¹⁸	Standardize interview dates by specialty/region to limit applicant choice
Interview day	Structured interviews (n = 11)¹³^,¹⁷^,¹⁸^,⁶³^,⁶⁴^,⁷³^,⁸⁵^,⁸⁶^,⁸⁹^,¹¹⁹^,¹²⁰	Interview using job-related and behavior-based questions
Interview day	Virtual reality scenarios (n = 1)¹²¹	Use virtual reality to assess communication and problem-solving skills
Post-interview communication	Post-interview communication ban (n = 7)⁵⁶^,¹¹³^,¹¹⁷^,¹²²^–¹²⁴	Impose a ban on all post-interview communications (eg, calls, emails)
Post-interview communication	Post-interview commitments (n = 1)¹²⁵	Require all post-interview commitments be documented in writing
Reforms to the Match Process
Match structure	Multiple match rounds (n = 8)¹⁸^,²²^-²⁴^,³⁴^,³⁵^,¹²⁶^,¹²⁷	Institute multiple match rounds (eg, early, regular, SOAP)
	Early assurance match (n = 6)¹⁷^,²⁹^,⁴⁷^,⁷⁰^,¹²⁶^,¹²⁷	Match early to residency at start of or during medical school
	Free market approach (n = 1)¹²⁸	Eliminate the Match, with applicants accepting positions on rolling basis

Open in a new tab

Abbreviations: EPAs, entrustable professional activities; USMLE, United States Medical Licensing Examination; AAMC, Association of American Medical Colleges; LORs, letters of recommendation; SOAP, Supplemental Offer and Acceptance Program.

Table 2.

Summation of Pros and Cons of Proposed Systems-Level Reforms to US Resident Selection Process, As Reported by Authors of 116 Articles (January 2005–February 2020)

Reform	Pros	Cons
Reforms to the Application Submission Process
Application cap	Address application volume; facilitate holistic review; cost savings	Difficult to determine optimal cap; deters competition
Supplemental applications	Limit application volume; signal program interest	Time-intensive; require additional review resources; generic writing
Signaling system	Gauge applicant interest; concentrate interview pool	Requires universal participation; incur additional fees/costs
Standardized program database	Identify suitable programs; tailor applications to programs	Applicants may misinterpret available information
Establish residency brand	Evaluate applicant/program compatibility; reduce resident attrition	Difficult and resource intensive to authentically capture brand
Reforms to the Application Review Process
Holistic review	Increase diversity; deemphasize USMLE examinations; improve “fit”	Resource-intensive to review; subjective; limited standardized data
Standardized grading	Address grade inflation; facilitate applicant comparison	Differing grading schemas between institutions
New medical school assessments	Promote holistic review; emphasize competency/professionalism	Difficult to validate; variable grading/objectives between schools
New medical school metrics	Uniform reporting of medical school experiences; enable comparison	Difficult to create metrics that encompass breadth of experiences
Residency selection examinations	Deemphasize USMLE examinations; facilitate holistic review	Subject applicants to additional testing; costly to develop/validate
Specialty-specific metrics	Facilitate holistic review; enable applicant comparisons	Subject applicants to additional testing; costly to develop/validate
Psychological assessments	Assess noncognitive attributes; facilitate holistic review	Subject to social desirability bias and Hawthorne effect; expensive
Personality assessments	Assess personality attributes; predict future behavior; improve “fit”	Subject to social desirability bias and Hawthorne effect; expensive
Require objective data in MSPE	Facilitate applicant comparison; increase data/metrics for review	Overemphasize numerical performance and class rank
Standardize MSPE structure	Enhance review efficiency; facilitate applicant comparison	None stated
Third-party review of MSPE	Improve AAMC guideline compliance; enhance review efficiency	Resource-intensive to review
Utilize SLOEs	Enhance review efficiency; facilitate applicant comparison; objective	Grade inflation without standardization or national norms
National SLOE norms	Curtail grade inflation	None stated
SLOE guidelines	Enhance uniformity of SLOE	None stated
Visual letters of recommendation	Enhance efficiency of review	None stated
Pre-interview screening	Assess early applicant/program “fit”; facilitate holistic review	Time- and resource-intensive
Standardize interview offer dates	Create predictable timeline; minimize clinical disruptions	None stated
Online interview scheduling	Preserve “first come, first served” interview scheduling	Clinical disruptions due to preoccupation with interview scheduling
Interview match	Reduce interview scheduling frenzy; discretely signal preferences	Limited applicant/program control over interview scheduling
Interview lottery	Reduce interview scheduling frenzy	Limited applicant/program control over interview scheduling
Interview cap	Distribute interviews more equitably; decrease travel costs	Limits applicant competition
Standardize interview dates	Limit interview overlap by same applicant cohort; decrease costs	Limits applicant competition; limits interview options
Structured interviews	Multidimensional evaluation; improved interrater reliability	Time- and resource-intensive training for interviewers
Virtual reality scenarios	Assess real-time communication and problem-solving skills	Time- and resource-intensive to create scenario
Post-interview communication ban	Improve Match integrity; minimize influence on final rank lists	None stated
Post-interview commitments	Improve accountability of commitments	None stated
Reforms to the Match Process
Multiple match rounds	Review interested applicants early; improve interview equity	Increased administrative burdens given multiple rounds
Early assurance match	Emphasize knowledge/skills; improve UME-GME transition	Requires early commitment; emphasizes medical school attended
Free market approach	Expedite medical training; decrease costs	Resource-intensive to determine graduation eligibility

Open in a new tab

Abbreviations: USMLE, United States Medical Licensing Examination; MSPE, Medical Student Performance Evaluation; AAMC, Association of American Medical Colleges; SLOE, Standardized Letter of Evaluation; UME, undergraduate medical education; GME, graduate medical education.

Reforms to the Application Submission Process

Application Cap:

Twenty-eight articles endorsed an application cap.⁴^,¹⁵^,¹⁸^,²¹^–⁴⁵ This was the most frequently proposed reform and implementation strategies varied. Many recommended fixed caps for all specialties, but there was no consensus on the optimal cap number.⁴^,¹⁵^,¹⁸^,²¹^–⁴¹ Others suggested variable caps based on the specific-specialty and/or applicant metrics,⁴¹^–⁴³ or “soft” caps imposed by resource-intensive supplemental applications or higher Electronic Residency Application Service (ERAS) fees, without explicitly setting limits.⁴¹^,⁴⁴^,⁴⁵ The cited advantages of application caps include cost-savings to applicants and decreased application volume for programs, presumably facilitating holistic review.⁵^,²⁴^,²⁶^–²⁸^,³⁹ Challenges include determining the optimal cap and limited program data currently available to inform applicant decisions.⁴⁰

Signaling Program Preference:

Eleven articles promoted supplemental applications to express program preference, including a program-specific paragraph in the personal statement, written or video statements of interest, or secondary applications.⁴^,¹⁸^,²⁶^–²⁸^,⁴⁰^,⁴¹^,⁴⁶^–⁴⁹ This proposal frequently co-occurred with application caps (n = 7) as a mechanism for reducing application volume through their time-intensive nature.²⁴^,⁵⁰^,⁵¹ However, requiring applicants to submit these materials results in an increase in resources needed for application review.⁵² Similarly, 5 articles endorsed the creation of a “signaling” system within ERAS or a separate, third-party platform, allowing applicants to designate a specified number of “preferred” programs.²⁴^,⁴⁴^,⁵⁰^–⁵² Such limited preference signaling makes applicant interest explicit and may facilitate holistic review by residency programs,⁴⁴ but may increase applicant costs via third-party servicing fees.⁵⁰

Standardized Program Database:

Twenty-one articles proposed the creation of a database with standardized program information⁴^,⁵^,¹⁵^,¹⁷^,¹⁸^,²⁴^,²⁷^–²⁹^,³³^,⁴¹^–⁴³^,⁵⁰^,⁵³^–⁶⁰ beyond data currently captured in the American Medical Association FREIDA Tool⁶¹ and Association of American Medical Colleges (AAMC) Residency Explorer Tool.⁶² Data captured may include program information (eg, curriculum, case logs, research opportunities, graduate outcomes), screening criteria (eg, USMLE scores, AOA status, DO/IMG status, publications), and metrics of previously matched applicants (eg, National Resident Matching Program [NRMP] statistical profiles). This proposal frequently co-occurred with application caps (n = 10) and supplemental applications (n = 4), as these reforms likely assist applicants in identifying suitable programs. Additionally, 6 articles suggested residency programs establish a “brand” (language used in the articles) that conveys the culture, desired attributes of residents, and ideal career path for graduates, thereby informing applicants of the type of candidate likely to thrive in that culture.¹³^,¹⁷^,³³^,⁴⁸^,⁶³^,⁶⁴ This reform may enhance the compatibility between applicants and programs, improving resident satisfaction, and decreasing residents leaving the program.⁶⁴

Reforms to the Application Review Process

Holistic Review:

Twelve articles advocated for holistic review with balanced consideration of academic performance, extracurricular activities, and personal experiences.¹⁸^,⁴⁰^,⁴¹^,⁶⁰^,⁶⁵^–⁷² One article promoted blinding Step 1 scores,⁶⁸ and another encouraged the removal of specialty board passage rates as part of the Accreditation Council for Graduate Medical Education accreditation of residency programs to facilitate holistic review.¹⁸ Additionally, 2 articles encouraged the creation of national norms to holistically quantify and compare applicant accomplishments.¹³^,⁶⁵ Holistic review may increase diversity and improve compatibility between applicants and programs, but requires additional resources and is subjective.⁴⁰^,⁶⁰^,⁷⁰ To mitigate these issues, 3 articles suggested standardized holistic applicant scoring generated from a weighted rubric.¹³^,⁷³^,⁷⁴

Medical School Grading, Examinations, and Metrics:

Two articles recommended nationally standardized grading to address grade inflation and facilitate applicant comparison.¹⁰^,⁷⁵ However, uniform grading is difficult to achieve across all institutions given differing grading schemas (eg, pass/fail or graded) and inconsistent language between—and even within—institutions depending upon the rotation or course.¹⁰^,⁷⁵ Four articles also proposed the creation of new medical school assessments, including competency-based evaluations, simulations, entrustable professional activities, and gateway exercises that serve as knowledge and skill checkpoints.⁴^,¹⁸^,⁶⁶^,⁷⁶ These assessments permit longitudinal evaluation of applicant performance, identify applicant strengths and weaknesses, and provide a common framework for applicant assessments.⁴⁷^,⁶⁶ However, concerns regarding validity coupled with variability in assessment utilization, learning objectives, and grading between institutions may preclude direct applicant comparisons.⁶⁶ Finally, one article advocated for new medical school metrics capturing personal and professional experiences in a standardized manner to permit easier comparison of applicants.⁵

Residency Evaluations and Metrics:

One article promoted the creation of a general residency selection examination, separate from the USMLE series, that is intentionally designed to assess the knowledge and skills necessary for residency.¹⁷ Similarly, 3 articles endorsed the creation of new specialty-specific examinations and metrics generated from customized National Board of Medical Examiners specialty tests or aptitude tests to capture specialty knowledge and skills.⁶⁵^,⁶⁸^,⁷⁷ Though a new residency selection examination or novel specialty-specific examinations/metrics may facilitate holistic review by deemphasizing Step 1 and Step 2 CK, they subject applicants to additional tests and require costly development and validation.⁶⁸

Noncognitive Assessments:

Twenty-three articles promoted introduction of noncognitive assessments in resident selection, including personality assessments (n = 10),¹⁷^,⁷²^,⁷⁸^–⁸⁵ and other psychological assessments including situational judgement tests (n = 5),¹⁷^,⁶⁴^,⁷²^,⁸³^,⁸⁵ emotional intelligence tests (n = 2),⁸³^,⁸⁶ grit assessments (n = 2),⁸⁴^,⁸⁷ moral reasoning tests (n = 1),⁶⁷ and unspecified behavior-based questionnaires (n = 4).¹³^,⁸⁸^–⁹⁰ These measures may provide insight into applicants' future behavior in residency and are customizable, allowing programs to identify applicants with the best compatibility for their unique culture, thereby reducing resident attrition.⁷⁸^–⁸⁰^,⁸²^–⁸⁴ However, these assessments may be costly to validate and implement, and results may be skewed by social desirability bias and the Hawthorne effect.⁷⁸^,⁸²^,⁸⁶

MSPE:

Twelve articles proposed requiring objective data in the MSPE, particularly class rank, clerkship grades, and shelf examination scores.⁴^,¹¹^,¹²^,¹⁸^,³¹^,³⁹^,⁵⁹^,⁹¹^–⁹⁵ Proponents also sought disclosure of professionalism issues, academic difficulties, and leaves of absence. Additionally, 8 articles endorsed standardization of the MSPE structure, content, and language.¹¹^,¹²^,⁵⁹^,⁶⁰^,⁷⁰^,⁹³^–⁹⁵ These 2 reforms co-occurred 5 times, with the common feature being facilitation of efficient applicant comparisons. Other modifications included third-party MSPE review to ensure institutional compliance with AAMC guidelines as these are currently unenforced.⁵

Standardized Letters of Recommendation or Evaluation:

Twenty articles supported standardized letters of recommendation (SLOR) or evaluation (SLOE) to uniformly summarize applicants' academic and professional potential for a given specialty.¹¹^,¹³^,¹⁷^,²⁷^,³¹^,⁴⁸^,⁶⁰^,⁹⁰^,⁹⁶^–¹⁰⁷ Despite their interpretability, comparability, and objectivity, SLORs/SLOEs may be undermined by grade inflation.²⁷^,⁴⁸^,⁹⁰^,⁹⁹^,¹⁰¹^,¹⁰⁴ Consequently, 3 articles proposed the creation of nationally normed SLOR rubrics and a database comparing letter writer and program performance to national standards to curtail inflation.²⁷^,⁹⁰^,¹⁰⁶ Similarly, one article advocated for published letter writer guidelines detailing the preparation of SLORs/SLOEs, particularly the assignment of appropriate grading or ranking.¹⁰⁷ Finally, one article endorsed visual letters of recommendation (eg, word clouds), utilizing terms from the SLOR to enhance review efficiency.¹⁰⁸

Reforms to the Interview Process

Pre-Interview Screening:

Thirteen articles proposed pre-interview screening via standardized video interviews,⁶⁶^,⁶⁹^,⁸⁸^,¹⁰⁹^,¹¹⁰ preliminary videoconference or telephone interviews,²¹^,²⁶^,⁵⁷^,⁶⁴^,⁶⁸^,⁷⁶^,¹¹¹ or regional interviews²⁹^,¹¹² prior to on-site interviews. Such screening practices enable early assessment of applicant/program compatibility and reduce the interview cohort size, delivering cost-savings to applicants and programs alike.⁵⁷^,⁶⁴^,¹⁰⁹^,¹¹² However, concerns exist regarding the scalability and increased administrative burden of pre-interview screening.⁸⁸^,¹⁰⁹

Interview Allocation and Scheduling:

Seven articles suggested date standardization for interview offers with an acceptance window, creating a predictable timeline for applicants and minimizing clinical distractions.⁴^,¹³^,²²^,⁵³^,¹¹³^,¹¹⁴ Alternatively, one article promoted online interview scheduling to preserve “first-come, first-served” acceptances,⁵⁷ and one article proposed an interview match in which applicants and programs submit rank lists for interviews and are then “matched” to interview.¹¹⁵ More radically, 2 articles proposed an interview lottery, with one assigning interview dates from a rank lists of preferred dates,¹¹⁴ and the other randomly filling 50% of interview positions with applicants meeting minimum criteria.³¹

Additionally, 6 articles supported an interview cap that limits the number of interviews an applicant can accept and attend.²¹^,²⁹^,⁷⁶^,¹¹³^,¹¹⁶^,¹¹⁷ An interview cap may facilitate more equitable interview allocation and decrease costs, but the optimal limit is unclear given variable applicant circumstances and specialty competitiveness.¹⁴^,⁷⁶^,¹¹⁷ Five articles promoted date standardization for interviews by specialty or region to decrease costs and serve as a de facto interview cap that limits interview overlap by the same top-tier applicants.²⁹^,⁵⁹^,⁷⁶^,¹¹⁶^,¹¹⁸

Interview Day:

Eleven articles promoted structured interviews utilizing standardized job-related and behavior-based questions that are scored with a rubric.¹³^,¹⁷^,¹⁸^,⁶³^,⁶⁴^,⁷³^,⁸⁵^,⁸⁶^,⁸⁹^,¹¹⁹^,¹²⁰ Structured interviews enable multidimensional assessment and improve efficiency and interrater reliability, but require interviewer training and the development of validated questions and scoring rubrics.¹³^,⁸⁶^,¹¹⁹ Additionally, one article proposed virtual reality scenarios involving multiple applicants to assess real-time communication and problem-solving skills,⁸⁹ and another promoted the use of skills-based simulations on interview day to uniformly assess technical abilities and knowledge base.¹²¹

Post-Interview Communication:

Six studies supported a ban on post-interview communication (eg, calls, emails, and “second looks”) and creation of an anonymous reporting system for violations.⁵⁶^,¹¹³^,¹¹⁷^,¹²²^–¹²⁴ This ban may enhance the integrity of the resident selection process and minimize undue influence on applicant rank lists.¹⁴^,¹²²^,¹²⁴ Recognizing the challenges of a moratorium, one article suggested allowing post-interview commitments with the requirement of written documentation to improve accountability.¹²⁵

Reforms to the Match Process

Eight articles proposed multiple match rounds with varying application caps per round.¹⁸^,²²^–²⁴^,³⁴^,³⁵^,¹²⁶^,¹²⁷ By limiting the number of applications received, this approach facilitates an in-depth review of truly interest applicants and may allocate interviews more equitably, as early matching of competitive applicants affords interview opportunities for other applicants in subsequent rounds.²⁴^,³⁴ Six articles supported an early assurance match, including guaranteed residency positions as a condition of medical school acceptance, “pre-matching” to home programs or programs where subinternships are performed, or allowing early acceptance to a consortium of institutions.¹⁷^,²⁹^,⁴⁷^,⁷⁰^,¹²⁶^,¹²⁷ This reform emphasizes knowledge and skill acquisition during medical school rather than residency securement and may improve the undergraduate to graduate medical education transition.⁴^,²⁹^,⁷⁰^,¹²⁶ However, moving up the resident selection decision may unnecessarily increase emphasis on the medical school an applicant attends.⁷⁰^,¹²⁶ Finally, one study endorsed a “free market” approach in which graduation-eligible applicants interview and accept residency positions on a rolling basis.¹²⁸ This approach may expedite medical training, but a continual reassessment of graduation readiness is burdensome, and rolling offers may pressure applicants into making decisions with incomplete knowledge.¹²⁸^,¹²⁹

Discussion

Calls for resident selection reform grew over the past 15 years, particularly among competitive specialties such as otolaryngology and orthopaedic surgery. This is likely in response to increasing application volume and applicant competitiveness with limited comparative metrics. Many popular reforms, including application caps, supplemental applications, and standardized letters of evaluation, seem to benefit both applicants and programs via application reduction and efficient applicant comparisons, facilitating holistic review.

Prior to the 2020–2021 application cycle, specialties implemented several reforms identified in this scoping review. For instance, emergency medicine instituted the SLOE in 1997 for efficient applicant comparison.⁹¹ Though initially hamstrung by grade inflation, a concerted effort to create a national cohort of experienced authors addressed this limitation, and the SLOE remains a mainstay of the emergency medicine resident selection process.¹⁰⁶ Likewise, emergency medicine partnered with the AAMC from 2017–2020 to pilot the standardized video interview, an asynchronous online interview that assesses applicants' professionalism and interpersonal communication skills.¹¹⁰ However, concerns regarding its validity, cost, and negative applicant perceptions prompted discontinuation of the standardized video interview.¹³⁰ Additionally, otolaryngology mandated a program-specific paragraph in the personal statement in 2015 to gauge applicant interest and deter application inflation.⁴⁶ Program-specific paragraphs were found to be generically crafted,⁴⁶ and seemingly exacerbated a downturn in the number of otolaryngology applicants, resulting in them now being optional.¹³¹^,¹³²

The COVID-19 pandemic has affected the residency application process in numerous ways,¹³³^–¹³⁶ accelerating calls for disruptive innovation and affording opportunities for novel reform.¹^,¹³⁷^–¹⁴¹ Application caps were the most frequently proposed reform (n = 28 articles), and many specialties have recommended limits for the 2020–2021 cycle.¹³⁸^,¹⁴² Some have suggested deriving application caps from the AAMC Apply Smart tool, which correlates application data with specialty entrance rates to identify the point of diminishing returns for application submissions stratified by applicant type (MD/DO/IMG) and Step 1 score tertile.¹⁴³ However, several methodologic concerns exist regarding the calculation and use of specialty entrance rates rather than match rates as well as the applicability of the data to individual applicants.¹⁴⁴ Moreover, the acceptability of application caps varies by applicant and specialty,²⁴^,¹⁴⁵ and unenforced recommendations are unlikely to alter applicant behavior.

In contrast to applications, an applicant's number of contiguous ranks, a proxy for the number of interviews attended, correlates well with their match rate.¹⁴⁶ A small cohort of highly competitive applicants accept and attend a disproportionate number of interviews, and virtual interviews are likely to exacerbate this maldistribution by removing cost and travel constraints.¹⁴⁷ Calls for interview caps are growing,¹⁴⁸^,¹⁴⁹ and ophthalmology utilized a centralized scheduling platform to limit applicants to 20 virtual inteviews.¹⁵⁰ Likewise, obstetrics and gynecology, orthopaedic surgery, and dermatology implemented standardized interview offer dates with acceptance windows, creating a predictable timeline.¹⁵¹^–¹⁵³

In addition, otolaryngology implemented preference signaling, permitting applicants 5 signals each to explicitly designate their interest in programs.¹⁵⁴ Coupling preference signaling with a comprehensive database of residency program information, as is underway by ophthalmology and obstetrics and gynecology, assists applicants in identifying suitable programs.¹⁵⁰^,¹⁵¹ Other proposed strategies such as pre-interview screening provide innovative approaches for narrowing the applicant pool before full virtual interviews, but risk introducing an unvalidated metric susceptible to implicit bias. Finally, 9 specialties promoted holistic review, which is easy to suggest, but difficult to achieve given that graduate medical education programs are under-resourced for the current application volume.¹⁴¹ Obstetrics and gynecology is further promoting holistic review via development of new application review metrics, an applicant compatibility index, and an early match program.¹⁵¹

Despite myriad proposed reforms, changes to the resident selection process have occurred piecemeal in single specialties. Articles in specialty-specific journals and lack of a common language impede widespread change. The fragmented nature of graduate medical education, both within and across specialties, further hinders progress. Additionally, multiple stakeholders (eg, AAMC, NRMP, San Francisco Match, Urology Match) are involved in the resident selection process and their agreement is requisite for national change. The Coalition for Physician Accountability has convened a cross-organization committee for this purpose, with recommendations expected in Spring 2021.¹⁵⁵ Careful examination of applicant and program experiences as well as match outcomes following implementation of these reforms is imperative to inform future directions.

This scoping review has several limitations, namely the potential exclusion of reforms published in editorials or commentaries without a title or abstract, rendering them difficult to identify via database queries. This likelihood was minimized by manual searching of leading undergraduate and graduate medical education journals and bibliographies of included studies. Additionally, articles describing novel reforms that lacked generalizability were excluded.

Conclusions

This scoping review characterized proposed reforms to the US resident selection process, developing a common language and framework to facilitate national conversations and change. The COVID-19 pandemic prompted many specialties to implement novel reforms identified in this review.

Supplementary Material

Click here for additional data file.^{(342.3KB, pdf)}

Acknowledgments

The authors would like to thank Lily Martin, MLIS, Icahn School of Medicine at Mount Sinai, for her assistance in designing the search strategies.

Footnotes

Funding: The authors report no external funding source for this study.

Conflict of interest: The authors declare they have no competing interests.

References

1.Hammoud MM, Standiford T, Carmody B. Potential implications of COVID-19 for the 2020-2021 residency application cycle. JAMA. 2020;324(1):29–30. doi: 10.1001/jama.2020.8911. [DOI] [PubMed] [Google Scholar]
2.National Resident Matching Program. Results of the 2011 NRMP applicant survey by preferred specialty and applicant type. 2021 https://www.nrmp.org/main-residency-match-data/ Accessed March 16.
3.National Resident Matching Program. Results of the 2019 NRMP applicant survey by preferred specialty and applicant type. 2021 https://www.nrmp.org/main-residency-match-data/ Accessed March 16.
4.Pereira AG, Chelminski PR, Chheda SG, et al. Application inflation for internal medicine applicants in the match: drivers, consequences, and potential solutions. Am J Med. 2016;129(8):885–891. doi: 10.1016/j.amjmed.2016.04.001. [DOI] [PubMed] [Google Scholar]
5.Katsufrakis PJ, Chaudhry HJ. Improving residency selection requires close study and better understanding of stakeholder needs. Acad Med. 2019;94(3):305–308. doi: 10.1097/ACM.0000000000002559. [DOI] [PubMed] [Google Scholar]
6.Williams M, Kim EJ, Pappas K, et al. The impact of United States Medical Licensing Exam (USMLE) Step 1 cutoff scores on recruitment of underrepresented minorities in medicine: a retrospective cross-sectional study. Health Sci Rep. 2020;3(2):1–8. doi: 10.1002/hsr2.161. [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Wagner JG, Schneberk T, Zobrist M, et al. Association between resident performance, rank list position, and United States Medical Licensing Exam Step 1 scores. J Emerg Med. 2017;52(3):332–340. doi: 10.1016/j.emergmed.2016.11.008. [DOI] [PubMed] [Google Scholar]
8.Harfmann KL, Zirwas MJ. Can performance in medical school predict performance in residency? A compilation and review of correlative studies. J Am Acad Dermatol. 2011;65(5):1010–1022. doi: 10.1016/j.aad.2010.07.034. [DOI] [PubMed] [Google Scholar]
9.Makhoul AT, Pontell ME, Kumar NG, Drolet BC. Objective measures needed—program directors' perspectives on a pass/fail USMLE Step 1. N Engl J Med. 2020;382(25):2389–2392. doi: 10.1056/NEJMp2006148. [DOI] [PubMed] [Google Scholar]
10.Westerman ME, Boe C, Bole R, et al. Evaluation of medical school grading variability in the United States: are all honors the same? Acad Med. 2019;94(12):1939–1945. doi: 10.1097/ACM. 0000000000002843. [DOI] [PubMed] [Google Scholar]
11.Andolsek KM. Improving the medical student performance evaluation to facilitate resident selection. Acad Med. 2016;91(11):1475–1479. doi: 10.1097/ACM.0000000000001386. [DOI] [PubMed] [Google Scholar]
12.Hom J, Richam I, Hall P, et al. The state of medical student performance evaluations: improved transparency or continued obfuscation. Acad Med. 2016;91(11):1534–1539. doi: 10.1097/ACM.0000000000001034. [DOI] [PubMed] [Google Scholar]
13.Lee AG, Golnik KC, Oetting TA, et al. Re-engineering the resident application selection process in ophthalmology: a literature review and recommendations for improvement. Surv Ophthalmol. 2008;53(2):164–176. doi: 10.1016/j.survophthal.2007.12.007. [DOI] [PubMed] [Google Scholar]
14.Frush BW, Byerley J. High-value interviewing: a call for quality improvement in the match process. Acad Med. 2019;94(3):324–327. doi: 10.1097/ACM.0000000000002545. [DOI] [PubMed] [Google Scholar]
15.Christophel JJ, Levin PA. Too much of a good thing. JAMA Otolaryngol Head Neck Surg. 2014;140(4):291–292. doi: 10.1001/jamaoto.2013.6647. [DOI] [PubMed] [Google Scholar]
16.Cummings M. Osteopathic students' graduate medical education aspirations versus realities: the relationship of osteopathic medicine and primary care. Acad Med. 2016;91(1):36–41. doi: 10.1097/ACM.0000000000000892. [DOI] [PubMed] [Google Scholar]
17.Andolsek KM. One small step for Step 1. Acad Med. 2019;94(3):309–313. doi: 10.1097/ACM.0000000000002560. [DOI] [PubMed] [Google Scholar]
18.Berger JS, Cioletti A. Viewpoint from 2 graduate medical education deans: application overload in the residency match process. J Grad Med Educ. 2016;8(3):317–321. doi: 10.4300/JGME-D-16-00239.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–473. doi: 10.7326/M18-0850. [DOI] [PubMed] [Google Scholar]
20.Cho JY, Lee EH. Reducing confusion about grounded theory and qualitative content analysis: similarities and differences. Qual Rep. 2014;19(32):1–20. [Google Scholar]
21.Gardner AK, Smink DS, Scott BG, Korndorffer JR, Jr, Harrington D, Ritter EM. How much are we spending on resident selection? J Surg Educ. 2018;75(6):e85–e90. doi: 10.1016/j.jsurg.2018.10.001. [DOI] [PubMed] [Google Scholar]
22.Arnold L, Sullivan C, Okah FA. A free-market approach to the Match: a proposal whose time has not yet come. Acad Med. 2018;93(1):16–19. doi: 10.1097/ACM.0000000000001782. [DOI] [PubMed] [Google Scholar]
23.Hueston WJ. A proposal to address the increasing number of residency applications. Acad Med. 2017;92(7):896–897. doi: 10.1097/ACM.0000000000001740. [DOI] [PubMed] [Google Scholar]
24.Ward M, Pingree C, Laury AM, Bowe SN. Applicant perspectives on the otolaryngology residency application process. JAMA Otolaryngol Head Neck Surg. 2017;143(8):782–787. doi: 10.1001/jamaoto.2017.0231. [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Nasreddine AY, Gallo R. Applying to orthopaedic residency and matching rates: analysis and review of the past 25 years. J Bone Joint Surg Am. 2019;101(24):e134. doi: 10.2106/JBJS.18.00371. [DOI] [PubMed] [Google Scholar]
26.Li NY, Gruppuso PA, Kalagara S, Eltorai AEM, DePasse JM, Daniels AH. Critical assessment of the contemporary orthopaedic surgery residency application process. J Bone Joint Surg Am. 2018;101(21):e114. doi: 10.2106/JBJS.18.00587. [DOI] [PubMed] [Google Scholar]
27.Trikha R, Keswani A, Ishmael CR, Greig D, Kelley BV, Bernthal NM. Current trends in orthopaedic surgery residency applications and match rates. J Bone Joint Surg Am. 2020;102(6):e24. doi: 10.2106/JBJS.19.00930. [DOI] [PubMed] [Google Scholar]
28.Naclerio RM, Pinto JM, Baroody FM. Drowning in applications for residency training: a program's perspective and simple solutions. JAMA Otolaryngol Head Neck Surg. 2014;140(8):695–696. doi: 10.1001/jamaoto.2014.1127. [DOI] [PubMed] [Google Scholar]
29.Agarwal N, Choi PA, Okonkwo DO, Barrow DL, Friedlander RM. Financial burden associated with the residency match in neurological surgery. J Neurosurg. 2017;126(1):184–190. doi: 10.3171/2015.12.JNS15488. [DOI] [PubMed] [Google Scholar]
30.Kraeutler MJ. It is time to change the status quo: limiting orthopedic surgery residency applications. Orthopedics. 2017;40(5):267–268. doi: 10.3928/01477447-20170928-01. [DOI] [PubMed] [Google Scholar]
31.Bernstein J. Not the last word: roll them bones—selecting orthopaedic surgery residents by lottery. Clin Orthop Relat Res. 2019;477(12):2635–2638. doi: 10.1097/CORR.0000000000001033. [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Weissbart SJ, Kim SJ, Feinn RS, Stock JA. Relationship between the number of residency applications and the yearly match rate: time to start thinking about an application limit? J Grad Med Educ. 2015;7(1):81–85. doi: 10.4300/JGME-D-14-00270.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Zhao H, Freedman A, Lerman S. Reforming the urology match application process: a role for the residency programs. J Urol. 2020;203(1):44–45. doi: 10.1097/JU.0000000000000464. [DOI] [PubMed] [Google Scholar]
34.Monir JG. Reforming the Match: a proposal for a new 3-phase system. J Grad Med Educ. 2020;12(1):7–9. doi: 10.4300/JGME-D-19-00425.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
35.London DA. SOAP for everyone: an evolutionary development of the Match. Acad Med. 2017;92(6):730. doi: 10.1097/ACM.0000000000001702. [DOI] [PubMed] [Google Scholar]
36.Burbano FM, Yao A, Burish N, et al. Solving congestion in the plastic surgery match: a game theory analysis. Plast Reconstr Surg. 2019;143(2):634–639. doi: 10.1097/PRS.0000000000005254. [DOI] [PubMed] [Google Scholar]
37.Baroody FM, Pinto JM, Naclerio RM. Otolaryngology (urban) legend: the more programs to which you apply, the better the chances of matching. Arch Otolaryngol Head Neck Surg. 2008;134(10):1038. doi: 10.1001/archotol.134.10.1038. [DOI] [PubMed] [Google Scholar]
38.Weissbart SJ, Hall SJ, Fultz BR, Stock JA. The urology match as prisoner's dilemma: a game theory perspective. Urology. 2013;82(4):791–797. doi: 10.1016/j.urology.2013.04.061. [DOI] [PubMed] [Google Scholar]
39.Weissbart SJ, Stock JA, Wein AJ. Challenges facing program directors in the urology match. Urol Pract. 2016;3(6):486–492. doi: 10.1016/j.urpr.2015.10.008. [DOI] [PubMed] [Google Scholar]
40.Putnam-Pite D. Viewpoint from a former medical student/now intern playing the game—balancing numbers and intangibles in the orthopedic surgery match. J Grad Med Educ. 2016;8(3):311–313. doi: 10.4300/JGME-D-16-002356.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Katsufrakis PJ, Uhler TA, Jones LD. The residency application process: pursuing improved outcomes through better understanding of the issues. Acad Med. 2016;91(11):1483–1487. doi: 10.1097/ACM.0000000000001411. [DOI] [PubMed] [Google Scholar]
42.Kaplan AB, Riedy KN, Grundfast KM. Increasing competitiveness for an otolaryngology residency: where we are and concerns about the future. Otolaryngol Head Neck Surg. 2015;153(5):699–701. doi: 10.1177/0194599815593734. [DOI] [PubMed] [Google Scholar]
43.Deng F, Chen JX, Wesevich A. More transparency is needed to curb excessive residency applications. Acad Med. 2017;92(7):895–896. doi: 10.1097/ACM.0000000000001741. [DOI] [PubMed] [Google Scholar]
44.Whipple ME, Law AB, Bly RA. A computer simulation model to analyze the application process for competitive residency programs. J Grad Med Educ. 2019;11(1):30–35. doi: 10.4300/JGME-D-18-00397.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
45.Stoddard HA. More about the role of USMLE Step 1 scores in resident selection. Acad Med. 2016;91(11):1469–1470. doi: 10.1097/ACM.0000000000001400. [DOI] [PubMed] [Google Scholar]
46.Stepan KO, Kaul VF, Raquib AR, et al. An evaluation of the program-specific paragraph in the otolaryngology residency application. Laryngoscope. 2018;128(11):2508–2513. doi: 10.1002/lary.27553. [DOI] [PubMed] [Google Scholar]
47.Pereira AG, Williams CM, Angus SV. Disruptive innovation and the residency Match: the time is now. J Grad Med Educ. 2019;11(1):36–38. doi: 10.4300/JGME-D-19-01091. [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Porter SE, Razi AE, Ramsey TB. Novel strategies to improve resident selection by improving cultural fit: AOA critical issues. J Bone Joint Surg Am. 2017;99(22):e120. doi: 10.2106/JBJS.17.00225. [DOI] [PubMed] [Google Scholar]
49.Puscas L, Esclamado R. Use of a secondary essay in the residency application process. JAMA Otolaryngol Head Neck Surg. 2015;141(7):591–592. doi: 10.1001/jamaoto.2015.0677. [DOI] [PubMed] [Google Scholar]
50.Bernstein J. Not the last word: want to match in an orthopaedic surgery residency? Send a rose to the program director. Clin Orthop Relat Res. 2017;475(12):2845–2849. doi: 10.1007/s11999-017-5500-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
51.Chen JX, Deng F, Gray ST. Preference signaling in the national resident matching program. JAMA Otolaryngol Head Neck Surg. 2018;144(10):951. doi: 10.1001/jamaoto.2018.1558. [DOI] [PubMed] [Google Scholar]
52.Salehi PP, Benito D, Michaelides E. A novel approach to the national resident matching program—the star system. JAMA Otolaryngol Head Neck Surg. 2018;144(5):397–398. doi: 10.1001/jamaoto.2018.0068. [DOI] [PubMed] [Google Scholar]
53.Kozin ED, Setthi RV, Lehmann A, et al. Analysis of an online match discussion board: improving the otolaryngology-head and neck surgery match. Otolaryngol Head Neck Surg. 2015;152(3):458–464. doi: 10.1177/0194599814561187. [DOI] [PMC free article] [PubMed] [Google Scholar]
54.Cabrera-Muffly C, Chang CWD, Puscas L. Current interview trail metrics in the otolaryngology match. Otolaryngol Head Neck Surg. 2017;156(6):1097–1103. doi: 10.1177/0194599817690723. [DOI] [PubMed] [Google Scholar]
55.Gupta D, Kumar S. ERAS: can it be revamped? One point of view. J Grad Med Educ. 2016;8(3):467. doi: 10.4300/JGME-D-16-00015.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
56.Bernstein J. Not the last word: agonizing appropriately over the residency match rank list. Clin Orthop Relat Res. 2018;476(12):2309–2312. doi: 10.1097/CORR.0000000000000520. [DOI] [PMC free article] [PubMed] [Google Scholar]
57.Hariton E, Bortoletto P, Ayogu N. Residency interviews in the 21st century. J Grad Med Educ. 2016;8(3):322–324. doi: 10.4300/JGME-D-15-00501.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
58.Susarla SM, Swanson EW, Slezak S, Lifchez SD, Redett RJ. The perception and costs of the interview process for plastic surgery residency programs: can the process be streamlined? Plast Reconst Surg. 2017;139(1):302–309. doi: 10.1097/PRS.0000000000002912. [DOI] [PubMed] [Google Scholar]
59.Aagaard EM, Abaza M. The residency application process—burden and consequences. N Engl J Med. 2016;374(4):303–305. doi: 10.1056/NEJMp1510394. [DOI] [PubMed] [Google Scholar]
60.Gliatto P, Karani R. Viewpoint from 2 undergraduate medical education deans the residency application process: working well, needs fixing, or broken beyond repair? J Grad Med Educ. 2016;8(3):307–310. doi: 10.4300/JGME-D-16-00230.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
61.American Medical Association. FREIDA. 2021 https://freida.ama-assn.org/ Accessed March 16.
62.Association of American Medical Colleges. Residency Explorer Tool. 2021 https://www.residencyexplorer.org Accessed March 16.
63.Lee WT, Esclamado RM, Puscas L. Selecting among otolaryngology residency applicants to train as tomorrow's leaders. JAMA Otolaryngol Head Neck Surg. 2013;139(8):770–771. doi: 10.1001/jamaoto.2013.3982. [DOI] [PubMed] [Google Scholar]
64.Lyons J, Bingmer K, Ammori J, Marks J. Utilization of a novel program-specific evaluation tool results in a decidedly different interview pool than traditional application review. J Surg Educ. 2019;76(6):e110–e117. doi: 10.1016/j.jsurg.2019.10.007. [DOI] [PubMed] [Google Scholar]
65.Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States Medical Licensing Examination Step 1 scores in residency selection. Acad Med. 2016;91(1):12–15. doi: 10.1097/ACM.0000000000000855. [DOI] [PubMed] [Google Scholar]
66.Radabaugh CL, Hawkins RE, Welcher CM, et al. Beyond the United States Medical Licensing Examination Score: assessing competence for entering residency. Acad Med. 2019;94(7):983–989. doi: 10.01097/ACM.0000000000002728. [DOI] [PubMed] [Google Scholar]
67.Bohm KC, Van Heest T, Gioe TJ, Agel J, Johnson TC, Van Heest A. Assessment of moral reasoning skills in the orthopaedic surgery resident applicant. J Bone Joint Surg Am. 2014;96(17) doi: 10.2106/JBJS.M.00706. e:151. [DOI] [PubMed] [Google Scholar]
68.Chen A, Shinkai K. Rethinking how we select dermatology applicants—turning the tide. JAMA Dermatol. 2017;153(3):259–260. doi: 10.1001/jamadermatol.2016.4683. [DOI] [PubMed] [Google Scholar]
69.King A, Mayer C, Starnes C, Barringer K, Beier L, Sule H. Using the Association of American Medical Colleges standardized video interview in a holistic residency application review. Cureus. 2017;9(12):e1913. doi: 10.7759/cureus.1913. [DOI] [PMC free article] [PubMed] [Google Scholar]
70.Moynahan KF. The current use of United States Medical Licensing Examination Step 1 scores: holistic admissions and student well-being are in the balance. Acad Med. 2018;93(7):963–965. doi: 10.1097/ACM.0000000000002101. [DOI] [PubMed] [Google Scholar]
71.Jones RL, Burk-Rafel J. The Match: a numbers game. Acad Med. 2017;92(6):731. doi: 10.1097/ACM.0000000000001704. [DOI] [PubMed] [Google Scholar]
72.Bowe SN, Schmalbach CE, Laury AM. The state of the otolaryngology match: a review of applicant trends, “impossible” qualifications, and implications. Otolaryngol Head Neck Surg. 2017;156(6):985–990. doi: 10.1177/0194599817695804. [DOI] [PubMed] [Google Scholar]
73.Martin M, Salzberg L. Resident characteristics to evaluate during recruitment and interview. A Delphi study. Educ Prim Care. 2017;28(2):81–85. doi: 10.1080/14739879.2016.1266696. [DOI] [PubMed] [Google Scholar]
74.Turner NS, Shaughnessy WJ, Berg EJ, Larson DR, Hanssen AD. A quantitative composite scoring tool for orthopaedic residency screening and selection. Clin Orthop Relat Res. 2006;449:50–55. doi: 10.1097/01.blo.0000224042.84839.44. [DOI] [PubMed] [Google Scholar]
75.Weissman S. Why reforms must be made to make the dean's letter useful. Acad Med. 2013;88(4):434. doi: 10.1097/ACM.0b013e318285f352. [DOI] [PubMed] [Google Scholar]
76.Gruppuso PA, Adashi EY. Residency placement fever: is it time for a reevaluation? Acad Med. 2017;92(7):923–926. doi: 10.1097/ACM.0000000000001468. [DOI] [PMC free article] [PubMed] [Google Scholar]
77.Bernstein J. Not the last word: ending the residency application arms race—starting with the USMLE. Clin Orthop Relat Res. 2016;474(12):2571–2576. doi: 10.1007/s11999-016-5108-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
78.Lubelski D, Healy AT, Friedman A, Ferraris D, Benzel EC, Schlenk R. Correlation of personality assessments with standard selection criteria for neurosurgical residency applicants. J Neurosurg. 2016;125(4):986–994. doi: 10.3171/2015.7.JNS15880. [DOI] [PubMed] [Google Scholar]
79.Tornetta P, III, Jacobs JJ, Sterling RS, Kogan M, Fletcher KA. Friedman AMl. Personality assessment in orthopaedic surgery. AOA critical issues. J Bone Joint Surg Am. 2019;101(4):e13. doi: 10.2106/JBJS.18.00578. [DOI] [PubMed] [Google Scholar]
80.Phillips D, Egol KA, Maculatis MC, et al. Personality factors associated with resident performance: results from 12 Accreditation Council for Graduate Medical Education accredited orthopaedic surgery programs. J Surg Educ. 2018;75(1):122–131. doi: 10.1016/j.jsurg.2017.06.023. [DOI] [PubMed] [Google Scholar]
81.Merlo LJ, Matveevskii AS. Personality testing may improve resident selection in anesthesiology programs. Med Teach. 2009;31(12):e551–e554. doi: 10.3109/01421590903390593. [DOI] [PMC free article] [PubMed] [Google Scholar]
82.Friedman AM. Using organizational science to improve the resident selection process: an outsider's perspective. Am J Med Qual. 2016;31(5):486–488. doi: 10.1177/1062860615615669. [DOI] [PubMed] [Google Scholar]
83.Gardner AK, Dunkin BJ. Evaluation of validity evidence for personality, emotional intelligence, and situational judgement tests to identify successful residents. JAMA Surg. 2018;153(5):409–416. doi: 10.1001/jamasurg.2017.5013. [DOI] [PMC free article] [PubMed] [Google Scholar]
84.Hughes BD, Perone JA, Cummins CB, et al. Personality testing may identify applicants who will become successful in general surgery residency. J Surg Res. 2019;333:240–248. doi: 10.1016/j.jss.2018.08.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
85.Gardner AK, Grantcharov T, Dunkin BJ. The science of selection: using best practices from industry to improve success in surgery training. J Surg Educ. 2018;75(2):278–285. doi: 10.1016/j.jsurg.2017.07.010. [DOI] [PubMed] [Google Scholar]
86.Lin DT, Kannappan A, Lau JN. The assessment of emotional intelligence among candidates interviewing for general surgery residency. J Surg Educ. 2013;70(4):514–521. doi: 10.1016/j.jsurg.2013.03.010. [DOI] [PubMed] [Google Scholar]
87.Kurian EB, Desai VS, Turner NS, et al. Is grit the new fit? Assessing non-cognitive variables in orthopedic surgery trainees. J Surg Educ. 2019;76(4):924–930. doi: 10.1016/j.jsurg.2019.01.010. [DOI] [PubMed] [Google Scholar]
88.Breitkopf DM, Green IC, Hopkins MR, Torbenson VE, Camp CL, Turner NS., III Use of asynchronous video interviews for selecting obstetrics and gynecology residents. Obstet Gynecol. 2019;134(suppl 1):9–15. doi: 10.1097/AOG.0000000000003432. [DOI] [PubMed] [Google Scholar]
89.Crawford SB, Monks SM, Wells RN. Virtual reality as an interview technique in evaluation of emergency medicine applicants. AEM Educ Train. 2018;2(4):328–333. doi: 10.1002/aet2/10113. [DOI] [PMC free article] [PubMed] [Google Scholar]
90.Love JN, Smith J, Weizberg M, et al. Council of emergency medicine residency directors' standardized letter of recommendation: the program director's perspective. 2014;21(6):680–687. doi: 10.1111/acem.12384. [DOI] [PubMed] [Google Scholar]
91.Benzinger R. From the desk of the program director: show me the rankings! J Grad Med Educ. 2014;6(3):413–414. doi: 10.4300/JGME-D-13-00448.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
92.Boysen-Osborn M, Mattson J, Yanuck J, et al. Ranking practice variability in the medical student performance evaluation: so bad, it's “good”. Acad Med. 2016;91(11):1540–1545. doi: 10.1097/ACM.0000000000001180. [DOI] [PMC free article] [PubMed] [Google Scholar]
93.Eisenberg RL. Suggestions for improving the medical student performance evaluation. Acad Med. 2016;91(11):1468. doi: 10.1097/ACM.0000000000001398. [DOI] [PubMed] [Google Scholar]
94.Green MM, Sanguino SM, Thomas JX., Jr Standardizing and improving the content of the dean's letter. Virtual Mentor. 2012;14(12):1021–1026. doi: 10.1001/virtualmentor.2012.14.12.oped1-1212. [DOI] [PubMed] [Google Scholar]
95.Boysen-Osborn M, Yanuck J, Mattson J, et al. Who to interview? Low adherence by U.S. medical schools to medical student performance evaluation format makes resident selection difficult. West J Emerg Med. 2017;18(1):50–55. doi: 10.5811/westjem.2016.10.32233. [DOI] [PMC free article] [PubMed] [Google Scholar]
96.Messner A, Teng M, Shimahara E, et al. A case for the standardized letter of recommendation in otolaryngology residency selection. Laryngoscope. 2014;124(1):2–3. doi: 10.1002/lary.24476. [DOI] [PubMed] [Google Scholar]
97.Wang RF, Zhang M, Alloo A, Stasko T, Miller JE, Kaffenberger JA. Characterization of the 2016-2017 dermatology standardized letter of recommendation. J Clin Aesthet Dermatol. 2018;11(3):26–29. [PMC free article] [PubMed] [Google Scholar]
98.Coates WC. Choosing the right resident: implications of the new electronic emergency medicine standardized letter of evaluation. J Grad Med Educ. 2019;11(2):187–188. doi: 10.4300/JGME-D-19-00163.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
99.Field NC, Gullick MM, German JW. Selection of neurological surgery applicants and the value of standardized letters of evaluation: a survey of United States program directors. World Neurosurg. 2020;136:e342–e346. doi: 10.1016/j.wneu.2019.12.176. [DOI] [PubMed] [Google Scholar]
100.Perkins JN, Liang C, McFann K, Abaza MM, Streubel SO, Prager JD. Standardized letter of recommendation for otolaryngology residency selection. Laryngoscope. 2013;123(1):123–133. doi: 10.1002/lary.23866. [DOI] [PMC free article] [PubMed] [Google Scholar]
101.Friedman R, Fang CH, Hasbun J, et al. Use of standardized letters of recommendation for otolaryngology head and neck surgery residency and the impact of gender. Laryngoscope. 2017;127(12):2738–2745. doi: 10.1002/lary.26619. [DOI] [PubMed] [Google Scholar]
102.Bajwa JM, Yudkowsky R, Belli D, Vu NV, Park YS. Validity evidence for a residency admissions standardized assessment letter for pediatrics. Teach Learn Med. 2018;30(2):173–183. doi: 10.1080/10401334.2017.1367297. [DOI] [PubMed] [Google Scholar]
103.Love JN, Ronan-Bentle SE, Lane DR, Hegarty CB. The standardized letter of evaluation for postgraduate training: a concept whose time has come? Acad Med. 2016;91(11):1480–1482. doi: 10.1097/ACM.0000000000001352. [DOI] [PubMed] [Google Scholar]
104.Kominsky AH, Bryson PC, Benninger MS, Tierney WS. Variability ratings in the otolaryngology standardized letter of recommendation. Otolaryngol Head Neck Surg. 2016;154(2):287–293. doi: 10.1177/0194599815623525. [DOI] [PubMed] [Google Scholar]
105.Rajesh A, Rivera M, Asaad M, et al. What are we really looking for in a letter of recommendation? J Surg Educ. 2019;76(6):e118–e124. doi: 10.1016/j.jsurg.2019.06.008. [DOI] [PubMed] [Google Scholar]
106.Jackson JS, Bond M, Love JN, Hegarty C. Emergency medicine standardized letter of evaluation (SLOE): findings from the new electronic SLOE format. J Grad Med Educ. 2019;11(2):182–186. doi: 10.4300/JGME-D-18-00344.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
107.Kang HP, Robertson DM, Levine WN, Lieberman JR. Evaluating the standardized letter of recommendation form in applicants to orthopaedic surgery residency. J Am Acad Orthop Surg. 2020;28(19):814–422. doi: 10.5435/JAAOS-D-19-00423. [DOI] [PubMed] [Google Scholar]
108.Bayrak SB, Villwock JA, Villwock MR, Chiu AG, Sykes KJ. Using word clouds to re-envision letters of recommendation for residency applicants. Laryngoscope. 2019;129(9):2026–2030. doi: 10.1002/lary.27613. [DOI] [PubMed] [Google Scholar]
109.Bird SB, Hern HG, Blomkalns A, et al. Innovation in residency selection: the AAMC standardized video interview. Acad Med. 2019;94(10):1489–1497. doi: 10.1097/ACM.0000000000002705. [DOI] [PubMed] [Google Scholar]
110.Gallahue FE, Hiller KM, Bird SB, et al. The AAMC standardized video interview: reactions and use by residency programs during the 2018 application cycle. Acad Med. 2019;94(10):1506–1512. doi: 10.1097/ACM.0000000000002714. [DOI] [PubMed] [Google Scholar]
111.Melendez MM, Dobryansky M, Alizadeh K. Live online video interviews dramatically improve the plastic surgery residency application process. Plast Reconstr Surg. 2021;130(1):240–241. doi: 10.1097/PRS.0b013e3182550411. [DOI] [PubMed] [Google Scholar]
112.McMakin KK, Caputo FJ, Hoell NG, Trani J, Carpenter JP, Lombardi JV. Trends in the 10-year history of the vascular integrated residency match: more work, higher cost, same result. J Vasc Surg. 2020;72(1):298–303. doi: 10.1016/j.jvs.2019.10.066. doi:101016/j.jvs.2019.10.066. [DOI] [PubMed] [Google Scholar]
113.Frush BW, Byerley J. High-value interviewing: a call for quality improvement in the Match process. Acad Med. 2019;94(3):324–327. doi: 10.1097/ACM.0000000000002545. [DOI] [PubMed] [Google Scholar]
114.Luftig D. The residency interview scheduling process: unintended consequences and a proposal for change. J Grad Med Educ. 2015;7(1):134. doi: 10.4300/JGME-D-14-00603.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
115.Melcher ML, Wapnir I, Ashlagi I. May the interview be with you: signal your preferences. J Grad Med Educ. 2019;11(1):39–40. doi: 10.4300/JGME-D-19-00002.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
116.Bhalla V, Sykes K, Kraft SM, Chiu AG. Commentary on Bowe et al, “The state of the otolaryngology Match: a review of applicant trends, ‘impossible' qualifications, and implications”. Otolaryngol Head Neck Surg. 2018;158(2):217–218. doi: 10.1177/0194599817737277. [DOI] [PubMed] [Google Scholar]
117.Wood JS, David LR. Outcome analysis of factors impacting plastic surgery match. Ann Plast Surg. 2010;64(6):770–774. doi: 10.1097/SAP.0b013e3181b4bcf5. [DOI] [PubMed] [Google Scholar]
118.Claiborne JR, Crantford JC, Swett KR, David LR. The plastic surgery match: predicting success and improving the process. Ann Plast Surg. 2013;70(6):698–703. doi: 10.1097/SAP.0b013e31828587d3. [DOI] [PubMed] [Google Scholar]
119.Gardner AK, D'Onofrio BC, Dunkin BJ. Can we get faculty interviewers on the same page? An examination of a structured interview course for surgeons. J Surg Educ. 2018;75(1):72–77. doi: 10.1016/j.jsurg.2017.06.006. [DOI] [PubMed] [Google Scholar]
120.Nallasamy S, Uhler T, Nallasamy N, Tapino PJ, Volpe NJ. Ophthalmology resident selection: current trends in selection criteria and improving the process. Ophthalmology. 2010;117(5):1041–1047. doi: 10.1016/j.ophtha.2009.07.034. [DOI] [PubMed] [Google Scholar]
121.Cannon K, Hartsell Z, Ivanov I, et al. Perceptions of internal medicine residency program candidates on the use of simulation in the selection process. J Grad Med Educ. 2014;6(2):338–340. doi: 10.4300/JGME-D-13-00276.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
122.Nagarkar PA, Janis JE. Fixing the “Match”: how to play the game. J Grad Med Educ. 2012;4(2):142–147. doi: 10.4300/JGME-D-11-00292.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
123.Wu AJ, Vapiwala N, Chmura SJ, et al. Taking “the game” out of the Match: a simple proposal. Int J Radiat Oncol Biol Phys. 2015;93(5):945–958. doi: 10.1016/j.ijrobp.2015.09.039. [DOI] [PubMed] [Google Scholar]
124.Ramkumar PN, Navarro SM, Chughtai M, Haeberle HS, Taylor SA, Mont MA. The orthopaedic surgery residency application process: an analysis of the applicant experience. J Am Acad Orthop Surg. 2018;26(15):537–544. doi: 10.5435/JAAOS-D-16-00935. [DOI] [PubMed] [Google Scholar]
125.Chervenak FA, McCullough LB, Cefalo R. An ethical justification and policy for making commitments during computerized residency application processes: the matching program as a laboratory for needed reform. Adv Health Sci Educ Theory Pract. 2011;16(3):427–433. doi: 10.1007/s10459-009-9167-1. [DOI] [PubMed] [Google Scholar]
126.Hammoud MM, Andrews J, Skochelak SE. Improving the residency application and selection process: an optional early result acceptance program. JAMA. 2020;323(6):503–504. doi: 10.1001/jama.2019.21212. [DOI] [PubMed] [Google Scholar]
127.Wong BJ. Reforming the Match process—early decision plans and the case for a consortia match. JAMA Otolaryngol Head Neck Surg. 2016;142(8):727–728. doi: 10.1001/jamaoto.2016.1232. [DOI] [PubMed] [Google Scholar]
128.Ray C, Bishop SE, Dow AW. Rethinking the Match: a proposal for modern matchmaking. Acad Med. 2018;93(1):45–47. doi: 10.1097/ACM.0000000000001781. [DOI] [PubMed] [Google Scholar]
129.Alvin MD, Magid D. Free-market approach to the Match: AN unfair and unfeasible proposal. Acad Med. 2018;93(5):671–672. doi: 10.1097/ACM.0000000000002149. [DOI] [PubMed] [Google Scholar]
130.American Academy of Emergency Medicine. Joint letter to the AAMC on the standardized video interview (SVI) 2021 https://www.aaem.org/current-news/joint-letter-to-the-aamc-on-the-svi Accessed March 16.
131.Kramer S. Is the program-specific paragraph responsible for declining application numbers? A commentary. Otolaryngol Head Neck Surg. 2018;158(2):215–216. doi: 10.1177/0194599817751053. [DOI] [PubMed] [Google Scholar]
132.Bowe SN, Schmalbach CE, Laury AM. Regarding “is the program-specific paragraph responsible for declining application numbers? A commentary.”. Otolaryngol Head Neck Surg. 2018;158(6):1150–1151. doi: 10.1177/0194599818771912. [DOI] [PubMed] [Google Scholar]
133.Chretien KC, Raj JM, Abraham RA, et al. AAIM recommendations for the 2020-2021 internal medicine residency application cycle in response to the COVID-19 pandemic. Am J Med. 2020;133(10):1223–1226. doi: 10.1016/j.amjmed.2020.06.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
134.Aiyer AA, Granger CJ, McCormick KL, et al. The impact of COVID-19 on the orthopaedic surgery residency application process. J Am Acad Orthop Surg. 2020;28(15):e633–e641. doi: 10.5435/JAAOS-D-20-00557. [DOI] [PMC free article] [PubMed] [Google Scholar]
135.Rekawek P, Henry A, Moe J, Schlieve T, Panchal N. The COVID-19 pandemic: implications for the oral and maxillofacial surgery residency application process. 2020] doi: 10.1002/jdd.12310. [published online ahead of print July 22. J Dent Educ doi: [DOI] [PMC free article] [PubMed]
136.Xie DX, Hillel AT, Ward BK. Otolaryngology residency match during the COVID-19 pandemic: What happens next? 2020] doi: 10.1001/jamaoto.2020.1078. [published online ahead of print June 4. JAMA Otolaryngol Head Neck Surg doi: [DOI] [PubMed]
137.Harary M, Bergsneider M. Letter: approaches to mitigate impact of COVID-19 pandemic on neurosurgical residency application cycle. Neurosurgery. 2020;87(2):e212–e213. doi: 10.1093/neuros/nyaa176. [DOI] [PMC free article] [PubMed] [Google Scholar]
138.Pelletier-Bui A, Franzen D, Smith L, et al. COVID-19: a driver for disruptive innovation of the emergency medicine residency application process. West J Emerg Med. 2020;21(5):1105–1113. doi: 10.5811/westjem.2020.8.48234. [DOI] [PMC free article] [PubMed] [Google Scholar]
139.Haas MRC, He S, Sternberg K, et al. Reimagining residency selection: part 1—a practical guide to recruitment in the post-COVID-19 era. J Grad Med Educ. 2020;12(5):539–544. doi: 10.4300/JGME-D-20-00907.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
140.Sternberg K, Jordan J, Haas MRC, et al. Reimagining residency selection: part 2—a practical guide to interviewing in the post-COVID-19 era. J Grad Med Educ. 2020;12(5):545–549. doi: 10.4300/JGME-D-20-00911.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
141.Association of American Medical Colleges. Specialty response to COVID19. 2021 https://students-residents.aamc.org/applying-residency/article/specialty-response-covid-19/ Accessed March 16.
142.Gabrielson AT, Kohn JR, Sparks HT, Clifton MM, Kohn TP. Proposed changes to the 2021 residency application process in the wake of COVID-19. Acad Med. 2020;95(9):1346–1349. doi: 10.1097/ACM.0000000000003520. [DOI] [PMC free article] [PubMed] [Google Scholar]
143.Association of American Medical Colleges. Apply smart Data to consider when applying to residency. 2021 https://students-residents.aamc.org/applying-residency/filteredresult/apply-smart-data-consider-when-applying-residency Accessed March 16.
144.Carmody JB. Applying smarter: a critique of the AMMC apply smart tools. J Grad Med Educ. 2020;12(1):10–13. doi: 10.4300/JGME-D-19-00495.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
145.Sweet ML, Williams CM, Stewart E, et al. Internal medicine residency program responses to the increase of residency applications: differences by program type and characteristics. J Grad Med Educ. 2019;11(6):698–793. doi: 10.4300/JGME-D-19-00194.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
146.National Resident Matching Program. Charting outcomes in the Match Senior students of US medical schools 2020. 2021 https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2020/07/Charting-Outcomes-in-the-Match-2020_MD-Senior_final.pdf Accessed March 16.
147.Lee AH, Young P, Liao R, Yi PH, Reh D, Best SR. I dream of gini: quantifying inequality in otolaryngology residency interviews. Laryngoscope. 2019;129:627–633. doi: 10.1002/lary.27521. [DOI] [PMC free article] [PubMed] [Google Scholar]
148.Morgan HK, Winkel AF, Standiford T, et al. The case for capping residency interviews. J Surg Educ. 2020] doi: 10.1016/j.jsurg.2020.08.033. [published online ahead of print September 14. [DOI] [PMC free article] [PubMed]
149.Burk-Rafel J, Standiford T. A novel ticket system for capping residency interview numbers: reimagining interviews in the COVID-19 era. Acad Med. 2021;96(1):50–55. doi: 10.1097/ACM.0000000000003745. [DOI] [PubMed] [Google Scholar]
150.Quillen DA, Siatkowski RM, Feldon S. COVID-19 and the ophthalmology match. Ophthalmology. 2020;128(2):181–184. doi: 10.1016/j.ophtha.2020.07.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
151.American Medical Association. Murphy B. The Match process is packed with stress. OB-GYNs aim to fix it. 2021 https://www.ama-assn.org/education/improve-gme/match-process-packed-stress-ob-gyns-aim-fix-it Accessed March 16.
152.Dermatology residency program director information regarding the interview process during the 2020-2021 application cycle. 2021 https://www.dermatologyprofessors.org/files/5_Statement%20on%20Interviews_Oct2020.pdf Accessed March 16.
153.American Orthopaedic Association. Universal interview offer day. 2021 https://www.aoassn.org/programs Accessed March 16.
154.Chang CWD, Pletcher SD, Thorne MC, Malekzadeh S. Preference signaling for the otolaryngology interview market. Laryngoscope. 2021;131(3):e744–e745. doi: 10.1002/lary.29151. [DOI] [PubMed] [Google Scholar]
155.Coalition for Physician Accountability. Reviewing the transition from UME to GME. 2021 https://physicianaccountability.org/ume-gme Accessed March 16.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Click here for additional data file.^{(342.3KB, pdf)}

[i1949-8357-13-3-355-b01] 1.Hammoud MM, Standiford T, Carmody B. Potential implications of COVID-19 for the 2020-2021 residency application cycle. JAMA. 2020;324(1):29–30. doi: 10.1001/jama.2020.8911. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b02] 2.National Resident Matching Program. Results of the 2011 NRMP applicant survey by preferred specialty and applicant type. 2021 https://www.nrmp.org/main-residency-match-data/ Accessed March 16.

[i1949-8357-13-3-355-b03] 3.National Resident Matching Program. Results of the 2019 NRMP applicant survey by preferred specialty and applicant type. 2021 https://www.nrmp.org/main-residency-match-data/ Accessed March 16.

[i1949-8357-13-3-355-b04] 4.Pereira AG, Chelminski PR, Chheda SG, et al. Application inflation for internal medicine applicants in the match: drivers, consequences, and potential solutions. Am J Med. 2016;129(8):885–891. doi: 10.1016/j.amjmed.2016.04.001. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b05] 5.Katsufrakis PJ, Chaudhry HJ. Improving residency selection requires close study and better understanding of stakeholder needs. Acad Med. 2019;94(3):305–308. doi: 10.1097/ACM.0000000000002559. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b06] 6.Williams M, Kim EJ, Pappas K, et al. The impact of United States Medical Licensing Exam (USMLE) Step 1 cutoff scores on recruitment of underrepresented minorities in medicine: a retrospective cross-sectional study. Health Sci Rep. 2020;3(2):1–8. doi: 10.1002/hsr2.161. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b07] 7.Wagner JG, Schneberk T, Zobrist M, et al. Association between resident performance, rank list position, and United States Medical Licensing Exam Step 1 scores. J Emerg Med. 2017;52(3):332–340. doi: 10.1016/j.emergmed.2016.11.008. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b08] 8.Harfmann KL, Zirwas MJ. Can performance in medical school predict performance in residency? A compilation and review of correlative studies. J Am Acad Dermatol. 2011;65(5):1010–1022. doi: 10.1016/j.aad.2010.07.034. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b09] 9.Makhoul AT, Pontell ME, Kumar NG, Drolet BC. Objective measures needed—program directors' perspectives on a pass/fail USMLE Step 1. N Engl J Med. 2020;382(25):2389–2392. doi: 10.1056/NEJMp2006148. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b10] 10.Westerman ME, Boe C, Bole R, et al. Evaluation of medical school grading variability in the United States: are all honors the same? Acad Med. 2019;94(12):1939–1945. doi: 10.1097/ACM. 0000000000002843. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b11] 11.Andolsek KM. Improving the medical student performance evaluation to facilitate resident selection. Acad Med. 2016;91(11):1475–1479. doi: 10.1097/ACM.0000000000001386. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b12] 12.Hom J, Richam I, Hall P, et al. The state of medical student performance evaluations: improved transparency or continued obfuscation. Acad Med. 2016;91(11):1534–1539. doi: 10.1097/ACM.0000000000001034. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b13] 13.Lee AG, Golnik KC, Oetting TA, et al. Re-engineering the resident application selection process in ophthalmology: a literature review and recommendations for improvement. Surv Ophthalmol. 2008;53(2):164–176. doi: 10.1016/j.survophthal.2007.12.007. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b14] 14.Frush BW, Byerley J. High-value interviewing: a call for quality improvement in the match process. Acad Med. 2019;94(3):324–327. doi: 10.1097/ACM.0000000000002545. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b15] 15.Christophel JJ, Levin PA. Too much of a good thing. JAMA Otolaryngol Head Neck Surg. 2014;140(4):291–292. doi: 10.1001/jamaoto.2013.6647. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b16] 16.Cummings M. Osteopathic students' graduate medical education aspirations versus realities: the relationship of osteopathic medicine and primary care. Acad Med. 2016;91(1):36–41. doi: 10.1097/ACM.0000000000000892. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b17] 17.Andolsek KM. One small step for Step 1. Acad Med. 2019;94(3):309–313. doi: 10.1097/ACM.0000000000002560. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b18] 18.Berger JS, Cioletti A. Viewpoint from 2 graduate medical education deans: application overload in the residency match process. J Grad Med Educ. 2016;8(3):317–321. doi: 10.4300/JGME-D-16-00239.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b19] 19.Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–473. doi: 10.7326/M18-0850. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b20] 20.Cho JY, Lee EH. Reducing confusion about grounded theory and qualitative content analysis: similarities and differences. Qual Rep. 2014;19(32):1–20. [Google Scholar]

[i1949-8357-13-3-355-b21] 21.Gardner AK, Smink DS, Scott BG, Korndorffer JR, Jr, Harrington D, Ritter EM. How much are we spending on resident selection? J Surg Educ. 2018;75(6):e85–e90. doi: 10.1016/j.jsurg.2018.10.001. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b22] 22.Arnold L, Sullivan C, Okah FA. A free-market approach to the Match: a proposal whose time has not yet come. Acad Med. 2018;93(1):16–19. doi: 10.1097/ACM.0000000000001782. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b23] 23.Hueston WJ. A proposal to address the increasing number of residency applications. Acad Med. 2017;92(7):896–897. doi: 10.1097/ACM.0000000000001740. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b24] 24.Ward M, Pingree C, Laury AM, Bowe SN. Applicant perspectives on the otolaryngology residency application process. JAMA Otolaryngol Head Neck Surg. 2017;143(8):782–787. doi: 10.1001/jamaoto.2017.0231. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b25] 25.Nasreddine AY, Gallo R. Applying to orthopaedic residency and matching rates: analysis and review of the past 25 years. J Bone Joint Surg Am. 2019;101(24):e134. doi: 10.2106/JBJS.18.00371. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b26] 26.Li NY, Gruppuso PA, Kalagara S, Eltorai AEM, DePasse JM, Daniels AH. Critical assessment of the contemporary orthopaedic surgery residency application process. J Bone Joint Surg Am. 2018;101(21):e114. doi: 10.2106/JBJS.18.00587. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b27] 27.Trikha R, Keswani A, Ishmael CR, Greig D, Kelley BV, Bernthal NM. Current trends in orthopaedic surgery residency applications and match rates. J Bone Joint Surg Am. 2020;102(6):e24. doi: 10.2106/JBJS.19.00930. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b28] 28.Naclerio RM, Pinto JM, Baroody FM. Drowning in applications for residency training: a program's perspective and simple solutions. JAMA Otolaryngol Head Neck Surg. 2014;140(8):695–696. doi: 10.1001/jamaoto.2014.1127. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b29] 29.Agarwal N, Choi PA, Okonkwo DO, Barrow DL, Friedlander RM. Financial burden associated with the residency match in neurological surgery. J Neurosurg. 2017;126(1):184–190. doi: 10.3171/2015.12.JNS15488. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b30] 30.Kraeutler MJ. It is time to change the status quo: limiting orthopedic surgery residency applications. Orthopedics. 2017;40(5):267–268. doi: 10.3928/01477447-20170928-01. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b31] 31.Bernstein J. Not the last word: roll them bones—selecting orthopaedic surgery residents by lottery. Clin Orthop Relat Res. 2019;477(12):2635–2638. doi: 10.1097/CORR.0000000000001033. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b32] 32.Weissbart SJ, Kim SJ, Feinn RS, Stock JA. Relationship between the number of residency applications and the yearly match rate: time to start thinking about an application limit? J Grad Med Educ. 2015;7(1):81–85. doi: 10.4300/JGME-D-14-00270.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b33] 33.Zhao H, Freedman A, Lerman S. Reforming the urology match application process: a role for the residency programs. J Urol. 2020;203(1):44–45. doi: 10.1097/JU.0000000000000464. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b34] 34.Monir JG. Reforming the Match: a proposal for a new 3-phase system. J Grad Med Educ. 2020;12(1):7–9. doi: 10.4300/JGME-D-19-00425.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b35] 35.London DA. SOAP for everyone: an evolutionary development of the Match. Acad Med. 2017;92(6):730. doi: 10.1097/ACM.0000000000001702. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b36] 36.Burbano FM, Yao A, Burish N, et al. Solving congestion in the plastic surgery match: a game theory analysis. Plast Reconstr Surg. 2019;143(2):634–639. doi: 10.1097/PRS.0000000000005254. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b37] 37.Baroody FM, Pinto JM, Naclerio RM. Otolaryngology (urban) legend: the more programs to which you apply, the better the chances of matching. Arch Otolaryngol Head Neck Surg. 2008;134(10):1038. doi: 10.1001/archotol.134.10.1038. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b38] 38.Weissbart SJ, Hall SJ, Fultz BR, Stock JA. The urology match as prisoner's dilemma: a game theory perspective. Urology. 2013;82(4):791–797. doi: 10.1016/j.urology.2013.04.061. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b39] 39.Weissbart SJ, Stock JA, Wein AJ. Challenges facing program directors in the urology match. Urol Pract. 2016;3(6):486–492. doi: 10.1016/j.urpr.2015.10.008. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b40] 40.Putnam-Pite D. Viewpoint from a former medical student/now intern playing the game—balancing numbers and intangibles in the orthopedic surgery match. J Grad Med Educ. 2016;8(3):311–313. doi: 10.4300/JGME-D-16-002356.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b41] 41.Katsufrakis PJ, Uhler TA, Jones LD. The residency application process: pursuing improved outcomes through better understanding of the issues. Acad Med. 2016;91(11):1483–1487. doi: 10.1097/ACM.0000000000001411. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b42] 42.Kaplan AB, Riedy KN, Grundfast KM. Increasing competitiveness for an otolaryngology residency: where we are and concerns about the future. Otolaryngol Head Neck Surg. 2015;153(5):699–701. doi: 10.1177/0194599815593734. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b43] 43.Deng F, Chen JX, Wesevich A. More transparency is needed to curb excessive residency applications. Acad Med. 2017;92(7):895–896. doi: 10.1097/ACM.0000000000001741. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b44] 44.Whipple ME, Law AB, Bly RA. A computer simulation model to analyze the application process for competitive residency programs. J Grad Med Educ. 2019;11(1):30–35. doi: 10.4300/JGME-D-18-00397.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b45] 45.Stoddard HA. More about the role of USMLE Step 1 scores in resident selection. Acad Med. 2016;91(11):1469–1470. doi: 10.1097/ACM.0000000000001400. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b46] 46.Stepan KO, Kaul VF, Raquib AR, et al. An evaluation of the program-specific paragraph in the otolaryngology residency application. Laryngoscope. 2018;128(11):2508–2513. doi: 10.1002/lary.27553. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b47] 47.Pereira AG, Williams CM, Angus SV. Disruptive innovation and the residency Match: the time is now. J Grad Med Educ. 2019;11(1):36–38. doi: 10.4300/JGME-D-19-01091. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b48] 48.Porter SE, Razi AE, Ramsey TB. Novel strategies to improve resident selection by improving cultural fit: AOA critical issues. J Bone Joint Surg Am. 2017;99(22):e120. doi: 10.2106/JBJS.17.00225. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b49] 49.Puscas L, Esclamado R. Use of a secondary essay in the residency application process. JAMA Otolaryngol Head Neck Surg. 2015;141(7):591–592. doi: 10.1001/jamaoto.2015.0677. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b50] 50.Bernstein J. Not the last word: want to match in an orthopaedic surgery residency? Send a rose to the program director. Clin Orthop Relat Res. 2017;475(12):2845–2849. doi: 10.1007/s11999-017-5500-9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b51] 51.Chen JX, Deng F, Gray ST. Preference signaling in the national resident matching program. JAMA Otolaryngol Head Neck Surg. 2018;144(10):951. doi: 10.1001/jamaoto.2018.1558. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b52] 52.Salehi PP, Benito D, Michaelides E. A novel approach to the national resident matching program—the star system. JAMA Otolaryngol Head Neck Surg. 2018;144(5):397–398. doi: 10.1001/jamaoto.2018.0068. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b53] 53.Kozin ED, Setthi RV, Lehmann A, et al. Analysis of an online match discussion board: improving the otolaryngology-head and neck surgery match. Otolaryngol Head Neck Surg. 2015;152(3):458–464. doi: 10.1177/0194599814561187. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b54] 54.Cabrera-Muffly C, Chang CWD, Puscas L. Current interview trail metrics in the otolaryngology match. Otolaryngol Head Neck Surg. 2017;156(6):1097–1103. doi: 10.1177/0194599817690723. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b55] 55.Gupta D, Kumar S. ERAS: can it be revamped? One point of view. J Grad Med Educ. 2016;8(3):467. doi: 10.4300/JGME-D-16-00015.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b56] 56.Bernstein J. Not the last word: agonizing appropriately over the residency match rank list. Clin Orthop Relat Res. 2018;476(12):2309–2312. doi: 10.1097/CORR.0000000000000520. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b57] 57.Hariton E, Bortoletto P, Ayogu N. Residency interviews in the 21st century. J Grad Med Educ. 2016;8(3):322–324. doi: 10.4300/JGME-D-15-00501.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b58] 58.Susarla SM, Swanson EW, Slezak S, Lifchez SD, Redett RJ. The perception and costs of the interview process for plastic surgery residency programs: can the process be streamlined? Plast Reconst Surg. 2017;139(1):302–309. doi: 10.1097/PRS.0000000000002912. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b59] 59.Aagaard EM, Abaza M. The residency application process—burden and consequences. N Engl J Med. 2016;374(4):303–305. doi: 10.1056/NEJMp1510394. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b60] 60.Gliatto P, Karani R. Viewpoint from 2 undergraduate medical education deans the residency application process: working well, needs fixing, or broken beyond repair? J Grad Med Educ. 2016;8(3):307–310. doi: 10.4300/JGME-D-16-00230.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b61] 61.American Medical Association. FREIDA. 2021 https://freida.ama-assn.org/ Accessed March 16.

[i1949-8357-13-3-355-b62] 62.Association of American Medical Colleges. Residency Explorer Tool. 2021 https://www.residencyexplorer.org Accessed March 16.

[i1949-8357-13-3-355-b63] 63.Lee WT, Esclamado RM, Puscas L. Selecting among otolaryngology residency applicants to train as tomorrow's leaders. JAMA Otolaryngol Head Neck Surg. 2013;139(8):770–771. doi: 10.1001/jamaoto.2013.3982. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b64] 64.Lyons J, Bingmer K, Ammori J, Marks J. Utilization of a novel program-specific evaluation tool results in a decidedly different interview pool than traditional application review. J Surg Educ. 2019;76(6):e110–e117. doi: 10.1016/j.jsurg.2019.10.007. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b65] 65.Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States Medical Licensing Examination Step 1 scores in residency selection. Acad Med. 2016;91(1):12–15. doi: 10.1097/ACM.0000000000000855. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b66] 66.Radabaugh CL, Hawkins RE, Welcher CM, et al. Beyond the United States Medical Licensing Examination Score: assessing competence for entering residency. Acad Med. 2019;94(7):983–989. doi: 10.01097/ACM.0000000000002728. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b67] 67.Bohm KC, Van Heest T, Gioe TJ, Agel J, Johnson TC, Van Heest A. Assessment of moral reasoning skills in the orthopaedic surgery resident applicant. J Bone Joint Surg Am. 2014;96(17) doi: 10.2106/JBJS.M.00706. e:151. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b68] 68.Chen A, Shinkai K. Rethinking how we select dermatology applicants—turning the tide. JAMA Dermatol. 2017;153(3):259–260. doi: 10.1001/jamadermatol.2016.4683. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b69] 69.King A, Mayer C, Starnes C, Barringer K, Beier L, Sule H. Using the Association of American Medical Colleges standardized video interview in a holistic residency application review. Cureus. 2017;9(12):e1913. doi: 10.7759/cureus.1913. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b70] 70.Moynahan KF. The current use of United States Medical Licensing Examination Step 1 scores: holistic admissions and student well-being are in the balance. Acad Med. 2018;93(7):963–965. doi: 10.1097/ACM.0000000000002101. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b71] 71.Jones RL, Burk-Rafel J. The Match: a numbers game. Acad Med. 2017;92(6):731. doi: 10.1097/ACM.0000000000001704. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b72] 72.Bowe SN, Schmalbach CE, Laury AM. The state of the otolaryngology match: a review of applicant trends, “impossible” qualifications, and implications. Otolaryngol Head Neck Surg. 2017;156(6):985–990. doi: 10.1177/0194599817695804. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b73] 73.Martin M, Salzberg L. Resident characteristics to evaluate during recruitment and interview. A Delphi study. Educ Prim Care. 2017;28(2):81–85. doi: 10.1080/14739879.2016.1266696. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b74] 74.Turner NS, Shaughnessy WJ, Berg EJ, Larson DR, Hanssen AD. A quantitative composite scoring tool for orthopaedic residency screening and selection. Clin Orthop Relat Res. 2006;449:50–55. doi: 10.1097/01.blo.0000224042.84839.44. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b75] 75.Weissman S. Why reforms must be made to make the dean's letter useful. Acad Med. 2013;88(4):434. doi: 10.1097/ACM.0b013e318285f352. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b76] 76.Gruppuso PA, Adashi EY. Residency placement fever: is it time for a reevaluation? Acad Med. 2017;92(7):923–926. doi: 10.1097/ACM.0000000000001468. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b77] 77.Bernstein J. Not the last word: ending the residency application arms race—starting with the USMLE. Clin Orthop Relat Res. 2016;474(12):2571–2576. doi: 10.1007/s11999-016-5108-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b78] 78.Lubelski D, Healy AT, Friedman A, Ferraris D, Benzel EC, Schlenk R. Correlation of personality assessments with standard selection criteria for neurosurgical residency applicants. J Neurosurg. 2016;125(4):986–994. doi: 10.3171/2015.7.JNS15880. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b79] 79.Tornetta P, III, Jacobs JJ, Sterling RS, Kogan M, Fletcher KA. Friedman AMl. Personality assessment in orthopaedic surgery. AOA critical issues. J Bone Joint Surg Am. 2019;101(4):e13. doi: 10.2106/JBJS.18.00578. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b80] 80.Phillips D, Egol KA, Maculatis MC, et al. Personality factors associated with resident performance: results from 12 Accreditation Council for Graduate Medical Education accredited orthopaedic surgery programs. J Surg Educ. 2018;75(1):122–131. doi: 10.1016/j.jsurg.2017.06.023. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b81] 81.Merlo LJ, Matveevskii AS. Personality testing may improve resident selection in anesthesiology programs. Med Teach. 2009;31(12):e551–e554. doi: 10.3109/01421590903390593. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b82] 82.Friedman AM. Using organizational science to improve the resident selection process: an outsider's perspective. Am J Med Qual. 2016;31(5):486–488. doi: 10.1177/1062860615615669. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b83] 83.Gardner AK, Dunkin BJ. Evaluation of validity evidence for personality, emotional intelligence, and situational judgement tests to identify successful residents. JAMA Surg. 2018;153(5):409–416. doi: 10.1001/jamasurg.2017.5013. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b84] 84.Hughes BD, Perone JA, Cummins CB, et al. Personality testing may identify applicants who will become successful in general surgery residency. J Surg Res. 2019;333:240–248. doi: 10.1016/j.jss.2018.08.003. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b85] 85.Gardner AK, Grantcharov T, Dunkin BJ. The science of selection: using best practices from industry to improve success in surgery training. J Surg Educ. 2018;75(2):278–285. doi: 10.1016/j.jsurg.2017.07.010. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b86] 86.Lin DT, Kannappan A, Lau JN. The assessment of emotional intelligence among candidates interviewing for general surgery residency. J Surg Educ. 2013;70(4):514–521. doi: 10.1016/j.jsurg.2013.03.010. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b87] 87.Kurian EB, Desai VS, Turner NS, et al. Is grit the new fit? Assessing non-cognitive variables in orthopedic surgery trainees. J Surg Educ. 2019;76(4):924–930. doi: 10.1016/j.jsurg.2019.01.010. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b88] 88.Breitkopf DM, Green IC, Hopkins MR, Torbenson VE, Camp CL, Turner NS., III Use of asynchronous video interviews for selecting obstetrics and gynecology residents. Obstet Gynecol. 2019;134(suppl 1):9–15. doi: 10.1097/AOG.0000000000003432. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b89] 89.Crawford SB, Monks SM, Wells RN. Virtual reality as an interview technique in evaluation of emergency medicine applicants. AEM Educ Train. 2018;2(4):328–333. doi: 10.1002/aet2/10113. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b90] 90.Love JN, Smith J, Weizberg M, et al. Council of emergency medicine residency directors' standardized letter of recommendation: the program director's perspective. 2014;21(6):680–687. doi: 10.1111/acem.12384. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b91] 91.Benzinger R. From the desk of the program director: show me the rankings! J Grad Med Educ. 2014;6(3):413–414. doi: 10.4300/JGME-D-13-00448.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b92] 92.Boysen-Osborn M, Mattson J, Yanuck J, et al. Ranking practice variability in the medical student performance evaluation: so bad, it's “good”. Acad Med. 2016;91(11):1540–1545. doi: 10.1097/ACM.0000000000001180. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b93] 93.Eisenberg RL. Suggestions for improving the medical student performance evaluation. Acad Med. 2016;91(11):1468. doi: 10.1097/ACM.0000000000001398. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b94] 94.Green MM, Sanguino SM, Thomas JX., Jr Standardizing and improving the content of the dean's letter. Virtual Mentor. 2012;14(12):1021–1026. doi: 10.1001/virtualmentor.2012.14.12.oped1-1212. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b95] 95.Boysen-Osborn M, Yanuck J, Mattson J, et al. Who to interview? Low adherence by U.S. medical schools to medical student performance evaluation format makes resident selection difficult. West J Emerg Med. 2017;18(1):50–55. doi: 10.5811/westjem.2016.10.32233. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b96] 96.Messner A, Teng M, Shimahara E, et al. A case for the standardized letter of recommendation in otolaryngology residency selection. Laryngoscope. 2014;124(1):2–3. doi: 10.1002/lary.24476. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b97] 97.Wang RF, Zhang M, Alloo A, Stasko T, Miller JE, Kaffenberger JA. Characterization of the 2016-2017 dermatology standardized letter of recommendation. J Clin Aesthet Dermatol. 2018;11(3):26–29. [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b98] 98.Coates WC. Choosing the right resident: implications of the new electronic emergency medicine standardized letter of evaluation. J Grad Med Educ. 2019;11(2):187–188. doi: 10.4300/JGME-D-19-00163.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b99] 99.Field NC, Gullick MM, German JW. Selection of neurological surgery applicants and the value of standardized letters of evaluation: a survey of United States program directors. World Neurosurg. 2020;136:e342–e346. doi: 10.1016/j.wneu.2019.12.176. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b100] 100.Perkins JN, Liang C, McFann K, Abaza MM, Streubel SO, Prager JD. Standardized letter of recommendation for otolaryngology residency selection. Laryngoscope. 2013;123(1):123–133. doi: 10.1002/lary.23866. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b101] 101.Friedman R, Fang CH, Hasbun J, et al. Use of standardized letters of recommendation for otolaryngology head and neck surgery residency and the impact of gender. Laryngoscope. 2017;127(12):2738–2745. doi: 10.1002/lary.26619. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b102] 102.Bajwa JM, Yudkowsky R, Belli D, Vu NV, Park YS. Validity evidence for a residency admissions standardized assessment letter for pediatrics. Teach Learn Med. 2018;30(2):173–183. doi: 10.1080/10401334.2017.1367297. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b103] 103.Love JN, Ronan-Bentle SE, Lane DR, Hegarty CB. The standardized letter of evaluation for postgraduate training: a concept whose time has come? Acad Med. 2016;91(11):1480–1482. doi: 10.1097/ACM.0000000000001352. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b104] 104.Kominsky AH, Bryson PC, Benninger MS, Tierney WS. Variability ratings in the otolaryngology standardized letter of recommendation. Otolaryngol Head Neck Surg. 2016;154(2):287–293. doi: 10.1177/0194599815623525. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b105] 105.Rajesh A, Rivera M, Asaad M, et al. What are we really looking for in a letter of recommendation? J Surg Educ. 2019;76(6):e118–e124. doi: 10.1016/j.jsurg.2019.06.008. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b106] 106.Jackson JS, Bond M, Love JN, Hegarty C. Emergency medicine standardized letter of evaluation (SLOE): findings from the new electronic SLOE format. J Grad Med Educ. 2019;11(2):182–186. doi: 10.4300/JGME-D-18-00344.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b107] 107.Kang HP, Robertson DM, Levine WN, Lieberman JR. Evaluating the standardized letter of recommendation form in applicants to orthopaedic surgery residency. J Am Acad Orthop Surg. 2020;28(19):814–422. doi: 10.5435/JAAOS-D-19-00423. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b108] 108.Bayrak SB, Villwock JA, Villwock MR, Chiu AG, Sykes KJ. Using word clouds to re-envision letters of recommendation for residency applicants. Laryngoscope. 2019;129(9):2026–2030. doi: 10.1002/lary.27613. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b109] 109.Bird SB, Hern HG, Blomkalns A, et al. Innovation in residency selection: the AAMC standardized video interview. Acad Med. 2019;94(10):1489–1497. doi: 10.1097/ACM.0000000000002705. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b110] 110.Gallahue FE, Hiller KM, Bird SB, et al. The AAMC standardized video interview: reactions and use by residency programs during the 2018 application cycle. Acad Med. 2019;94(10):1506–1512. doi: 10.1097/ACM.0000000000002714. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b111] 111.Melendez MM, Dobryansky M, Alizadeh K. Live online video interviews dramatically improve the plastic surgery residency application process. Plast Reconstr Surg. 2021;130(1):240–241. doi: 10.1097/PRS.0b013e3182550411. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b112] 112.McMakin KK, Caputo FJ, Hoell NG, Trani J, Carpenter JP, Lombardi JV. Trends in the 10-year history of the vascular integrated residency match: more work, higher cost, same result. J Vasc Surg. 2020;72(1):298–303. doi: 10.1016/j.jvs.2019.10.066. doi:101016/j.jvs.2019.10.066. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b113] 113.Frush BW, Byerley J. High-value interviewing: a call for quality improvement in the Match process. Acad Med. 2019;94(3):324–327. doi: 10.1097/ACM.0000000000002545. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b114] 114.Luftig D. The residency interview scheduling process: unintended consequences and a proposal for change. J Grad Med Educ. 2015;7(1):134. doi: 10.4300/JGME-D-14-00603.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b115] 115.Melcher ML, Wapnir I, Ashlagi I. May the interview be with you: signal your preferences. J Grad Med Educ. 2019;11(1):39–40. doi: 10.4300/JGME-D-19-00002.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b116] 116.Bhalla V, Sykes K, Kraft SM, Chiu AG. Commentary on Bowe et al, “The state of the otolaryngology Match: a review of applicant trends, ‘impossible' qualifications, and implications”. Otolaryngol Head Neck Surg. 2018;158(2):217–218. doi: 10.1177/0194599817737277. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b117] 117.Wood JS, David LR. Outcome analysis of factors impacting plastic surgery match. Ann Plast Surg. 2010;64(6):770–774. doi: 10.1097/SAP.0b013e3181b4bcf5. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b118] 118.Claiborne JR, Crantford JC, Swett KR, David LR. The plastic surgery match: predicting success and improving the process. Ann Plast Surg. 2013;70(6):698–703. doi: 10.1097/SAP.0b013e31828587d3. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b119] 119.Gardner AK, D'Onofrio BC, Dunkin BJ. Can we get faculty interviewers on the same page? An examination of a structured interview course for surgeons. J Surg Educ. 2018;75(1):72–77. doi: 10.1016/j.jsurg.2017.06.006. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b120] 120.Nallasamy S, Uhler T, Nallasamy N, Tapino PJ, Volpe NJ. Ophthalmology resident selection: current trends in selection criteria and improving the process. Ophthalmology. 2010;117(5):1041–1047. doi: 10.1016/j.ophtha.2009.07.034. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b121] 121.Cannon K, Hartsell Z, Ivanov I, et al. Perceptions of internal medicine residency program candidates on the use of simulation in the selection process. J Grad Med Educ. 2014;6(2):338–340. doi: 10.4300/JGME-D-13-00276.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b122] 122.Nagarkar PA, Janis JE. Fixing the “Match”: how to play the game. J Grad Med Educ. 2012;4(2):142–147. doi: 10.4300/JGME-D-11-00292.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b123] 123.Wu AJ, Vapiwala N, Chmura SJ, et al. Taking “the game” out of the Match: a simple proposal. Int J Radiat Oncol Biol Phys. 2015;93(5):945–958. doi: 10.1016/j.ijrobp.2015.09.039. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b124] 124.Ramkumar PN, Navarro SM, Chughtai M, Haeberle HS, Taylor SA, Mont MA. The orthopaedic surgery residency application process: an analysis of the applicant experience. J Am Acad Orthop Surg. 2018;26(15):537–544. doi: 10.5435/JAAOS-D-16-00935. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b125] 125.Chervenak FA, McCullough LB, Cefalo R. An ethical justification and policy for making commitments during computerized residency application processes: the matching program as a laboratory for needed reform. Adv Health Sci Educ Theory Pract. 2011;16(3):427–433. doi: 10.1007/s10459-009-9167-1. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b126] 126.Hammoud MM, Andrews J, Skochelak SE. Improving the residency application and selection process: an optional early result acceptance program. JAMA. 2020;323(6):503–504. doi: 10.1001/jama.2019.21212. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b127] 127.Wong BJ. Reforming the Match process—early decision plans and the case for a consortia match. JAMA Otolaryngol Head Neck Surg. 2016;142(8):727–728. doi: 10.1001/jamaoto.2016.1232. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b128] 128.Ray C, Bishop SE, Dow AW. Rethinking the Match: a proposal for modern matchmaking. Acad Med. 2018;93(1):45–47. doi: 10.1097/ACM.0000000000001781. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b129] 129.Alvin MD, Magid D. Free-market approach to the Match: AN unfair and unfeasible proposal. Acad Med. 2018;93(5):671–672. doi: 10.1097/ACM.0000000000002149. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b130] 130.American Academy of Emergency Medicine. Joint letter to the AAMC on the standardized video interview (SVI) 2021 https://www.aaem.org/current-news/joint-letter-to-the-aamc-on-the-svi Accessed March 16.

[i1949-8357-13-3-355-b131] 131.Kramer S. Is the program-specific paragraph responsible for declining application numbers? A commentary. Otolaryngol Head Neck Surg. 2018;158(2):215–216. doi: 10.1177/0194599817751053. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b132] 132.Bowe SN, Schmalbach CE, Laury AM. Regarding “is the program-specific paragraph responsible for declining application numbers? A commentary.”. Otolaryngol Head Neck Surg. 2018;158(6):1150–1151. doi: 10.1177/0194599818771912. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b133] 133.Chretien KC, Raj JM, Abraham RA, et al. AAIM recommendations for the 2020-2021 internal medicine residency application cycle in response to the COVID-19 pandemic. Am J Med. 2020;133(10):1223–1226. doi: 10.1016/j.amjmed.2020.06.002. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b134] 134.Aiyer AA, Granger CJ, McCormick KL, et al. The impact of COVID-19 on the orthopaedic surgery residency application process. J Am Acad Orthop Surg. 2020;28(15):e633–e641. doi: 10.5435/JAAOS-D-20-00557. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b135] 135.Rekawek P, Henry A, Moe J, Schlieve T, Panchal N. The COVID-19 pandemic: implications for the oral and maxillofacial surgery residency application process. 2020] doi: 10.1002/jdd.12310. [published online ahead of print July 22. J Dent Educ doi: [DOI] [PMC free article] [PubMed]

[i1949-8357-13-3-355-b136] 136.Xie DX, Hillel AT, Ward BK. Otolaryngology residency match during the COVID-19 pandemic: What happens next? 2020] doi: 10.1001/jamaoto.2020.1078. [published online ahead of print June 4. JAMA Otolaryngol Head Neck Surg doi: [DOI] [PubMed]

[i1949-8357-13-3-355-b137] 137.Harary M, Bergsneider M. Letter: approaches to mitigate impact of COVID-19 pandemic on neurosurgical residency application cycle. Neurosurgery. 2020;87(2):e212–e213. doi: 10.1093/neuros/nyaa176. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b138] 138.Pelletier-Bui A, Franzen D, Smith L, et al. COVID-19: a driver for disruptive innovation of the emergency medicine residency application process. West J Emerg Med. 2020;21(5):1105–1113. doi: 10.5811/westjem.2020.8.48234. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b139] 139.Haas MRC, He S, Sternberg K, et al. Reimagining residency selection: part 1—a practical guide to recruitment in the post-COVID-19 era. J Grad Med Educ. 2020;12(5):539–544. doi: 10.4300/JGME-D-20-00907.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b140] 140.Sternberg K, Jordan J, Haas MRC, et al. Reimagining residency selection: part 2—a practical guide to interviewing in the post-COVID-19 era. J Grad Med Educ. 2020;12(5):545–549. doi: 10.4300/JGME-D-20-00911.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b141] 141.Association of American Medical Colleges. Specialty response to COVID19. 2021 https://students-residents.aamc.org/applying-residency/article/specialty-response-covid-19/ Accessed March 16.

[i1949-8357-13-3-355-b142] 142.Gabrielson AT, Kohn JR, Sparks HT, Clifton MM, Kohn TP. Proposed changes to the 2021 residency application process in the wake of COVID-19. Acad Med. 2020;95(9):1346–1349. doi: 10.1097/ACM.0000000000003520. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b143] 143.Association of American Medical Colleges. Apply smart Data to consider when applying to residency. 2021 https://students-residents.aamc.org/applying-residency/filteredresult/apply-smart-data-consider-when-applying-residency Accessed March 16.

[i1949-8357-13-3-355-b144] 144.Carmody JB. Applying smarter: a critique of the AMMC apply smart tools. J Grad Med Educ. 2020;12(1):10–13. doi: 10.4300/JGME-D-19-00495.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b145] 145.Sweet ML, Williams CM, Stewart E, et al. Internal medicine residency program responses to the increase of residency applications: differences by program type and characteristics. J Grad Med Educ. 2019;11(6):698–793. doi: 10.4300/JGME-D-19-00194.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b146] 146.National Resident Matching Program. Charting outcomes in the Match Senior students of US medical schools 2020. 2021 https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2020/07/Charting-Outcomes-in-the-Match-2020_MD-Senior_final.pdf Accessed March 16.

[i1949-8357-13-3-355-b147] 147.Lee AH, Young P, Liao R, Yi PH, Reh D, Best SR. I dream of gini: quantifying inequality in otolaryngology residency interviews. Laryngoscope. 2019;129:627–633. doi: 10.1002/lary.27521. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b148] 148.Morgan HK, Winkel AF, Standiford T, et al. The case for capping residency interviews. J Surg Educ. 2020] doi: 10.1016/j.jsurg.2020.08.033. [published online ahead of print September 14. [DOI] [PMC free article] [PubMed]

[i1949-8357-13-3-355-b149] 149.Burk-Rafel J, Standiford T. A novel ticket system for capping residency interview numbers: reimagining interviews in the COVID-19 era. Acad Med. 2021;96(1):50–55. doi: 10.1097/ACM.0000000000003745. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b150] 150.Quillen DA, Siatkowski RM, Feldon S. COVID-19 and the ophthalmology match. Ophthalmology. 2020;128(2):181–184. doi: 10.1016/j.ophtha.2020.07.012. [DOI] [PMC free article] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b151] 151.American Medical Association. Murphy B. The Match process is packed with stress. OB-GYNs aim to fix it. 2021 https://www.ama-assn.org/education/improve-gme/match-process-packed-stress-ob-gyns-aim-fix-it Accessed March 16.

[i1949-8357-13-3-355-b152] 152.Dermatology residency program director information regarding the interview process during the 2020-2021 application cycle. 2021 https://www.dermatologyprofessors.org/files/5_Statement%20on%20Interviews_Oct2020.pdf Accessed March 16.

[i1949-8357-13-3-355-b153] 153.American Orthopaedic Association. Universal interview offer day. 2021 https://www.aoassn.org/programs Accessed March 16.

[i1949-8357-13-3-355-b154] 154.Chang CWD, Pletcher SD, Thorne MC, Malekzadeh S. Preference signaling for the otolaryngology interview market. Laryngoscope. 2021;131(3):e744–e745. doi: 10.1002/lary.29151. [DOI] [PubMed] [Google Scholar]

[i1949-8357-13-3-355-b155] 155.Coalition for Physician Accountability. Reviewing the transition from UME to GME. 2021 https://physicianaccountability.org/ume-gme Accessed March 16.

PERMALINK

Systems-Level Reforms to the US Resident Selection Process: A Scoping Review

Ryley K Zastrow, BS

Jesse Burk-Rafel, MD, MRes

Daniel A London, MD, MS

Abstract

Background

Objective

Methods

Results

Conclusions

Introduction

Methods

Search Strategy

Inclusion and Exclusion Criteria

Screening and Full-Text Review

Data Extraction and Synthesis

Results

Figure.

Table 1.

Table 2.

Reforms to the Application Submission Process

Application Cap:

Signaling Program Preference:

Standardized Program Database:

Reforms to the Application Review Process

Holistic Review:

Medical School Grading, Examinations, and Metrics:

Residency Evaluations and Metrics:

Noncognitive Assessments:

MSPE:

Standardized Letters of Recommendation or Evaluation:

Reforms to the Interview Process

Pre-Interview Screening:

Interview Allocation and Scheduling:

Interview Day:

Post-Interview Communication:

Reforms to the Match Process

Discussion

Conclusions

Supplementary Material

Acknowledgments

Footnotes

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases