The Use of Artificial Intelligence in Residency Application Evaluation—A Scoping Review

Visual Abstract.

ABSTRACT

Background Several residency programs have begun investigating artificial intelligence (AI) methods to facilitate application screening processes. However, no unifying guidelines for these methods exist.

Objective We sought to perform a scoping review of AI model development and use in residency/fellowship application review, including if bias was explored.

Methods A scoping review was performed according to PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines where a systematic search strategy identified relevant literature within the databases MEDLINE, Embase, and Scopus from inception to September 29, 2023. No limitations on language, article type, or geographic affiliation were placed on the search parameters. Data were extracted from relevant documents, and study findings were synthesized by the authors.

Results Twelve studies met inclusion criteria. Most used AI to predict interviews or rank lists (9 of 12, 75%), while the remaining 3 articles (25%) evaluated letters of recommendation with natural language processing. Six articles (50%) compared the model’s output to a human-created rank list. Most of the reviewed articles (9 of 12, 75%) mention bias; however, few explicitly modeled biases by accounting for or examining the effect of demographic factors (3 of 12, 25%).

Conclusions Few studies have been published on incorporating AI into residency/fellowship selection, and bias remains largely unexplored. There is a need for standardization in bias and fairness reporting within this area of research.

Introduction

Residency and fellowship programs strive to use a holistic review to find the optimal pairing between applicants and programs.^1,2 However, holistic review’s widespread implementation has been limited by its time-consuming nature, constraints on resources, human biases, and the absence of a universally accepted methodology.^3-5 These challenges have been exacerbated by the surge in applicant submissions over the last 5 years.^6,7 Therefore, residency programs have returned to metric-based cutoffs to narrow the number of applicants considered by each program.^8-12 Consequently, across specialties, between 30% to 65% of residency applications encounter rejection before undergoing holistic review.¹³ This may inadvertently uphold inequitable practices in residency and fellowship recruitment.^14,15

In parallel, there has been rapid adoption and innovation within artificial intelligence (AI)—namely natural language processing, generative pre-trained transformers, and machine learning models—due to its ability to quickly aggregate and summarize large amounts of data. With AI’s mainstream ascendence, concerns have been raised about algorithmic bias and regulation.¹⁶ These concerns motivated the passing of regulatory legislation at multiple levels of government.^17-19 As a result, over 31 states have adopted resolutions or enacted legislation regarding the use of AI.²⁰

Within this broader context, several residency programs have investigated using generative AI methods to facilitate applicant screening with the primary aims of reducing the resources required to equitably screen all applications,^3,15,21 making application review more standardized,^22-25 identifying resident values in unstructured data,^26-28 and avoiding bias.^29,30 Institutions innovating with generative AI deserve recognition for their early efforts to utilize AI in practice, particularly because these methods will likely become a standard part of application review. However, there are growing concerns about the presence of bias, whether implicit or explicit, in the AI models incorporated into determining the future of physicians-in-training.¹⁶ For this reason, we aimed to perform a scoping review of English language manuscripts documenting how AI, broadly defined, has been developed for uses within residency or fellowship application review and how those models account for bias.

Methods

The use of AI in trainee application evaluation is a broad and understudied topic, making a scoping review the most suitable methodology for identifying key trends and drawing overall conclusions within the field. The scoping review was performed according to published PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Scoping Reviews) guidelines.³¹ A systematic search strategy identified relevant literature. Data was extracted from eligible documents, and findings were synthesized. Results have been reported in alignment with the PRISMA-ScR Checklist (online supplementary data).

Search Strategy

A medical librarian with expertise in systematic searching composed a sensitive search utilizing a mix of keywords and subject headings that represented the concepts of recruitment, residency, and artificial intelligence. A second medical librarian peer-reviewed the search strategy in accordance with the modified PRESS (Peer Review of Electronic Search Strategies) checklist.³² No limitations on language, article type, or geographic affiliation were placed on the search parameters. The databases MEDLINE, Embase, and Scopus were searched from inception to September 29, 2023. All search results were compiled in EndNote (Clarivate) and imported into Covidence for deduplication and subsequent screening. The complete search strategies are shown in the online supplementary data.

Study Selection: Title and Abstract Screening

Articles were deemed eligible for this study if they met the following criteria: (1) reported on research; (2) mentioned or alluded to residency recruitment or selection; and (3) mentioned or alluded to AI. Articles that solely discussed AI in medical school admissions and did not comment on residency recruitment were excluded from this study. A single study member (M.D.S.) evaluated all titles and abstracts identified by the database screen for eligibility.

Study Selection: Full-Text Screening

The articles identified as eligible in the preliminary title and abstract screen underwent full-text screening by 2 study members (M.D.S., A.L.S.). The reviewers independently screened the full text of each article through Covidence. Any disagreements were resolved through discussion and the contribution of a third reviewer (T.C.H.). During this stage of review, an article on the use of ChatGPT (OpenAI) by residency applicants met the original inclusion criteria but was excluded from this study. This article was determined to be outside of the study scope because it did not describe how residency programs utilized AI for applicant evaluation. All other articles meeting eligibility criteria during full-text screen were marked for data extraction.

Data Extraction and Synthesis

The data extraction sheet was developed iteratively to ensure relevant and consistent information capture. Three articles were randomly selected for pilot testing of the data extraction sheet, and revisions were made as needed. Eligible articles were then randomly assigned to 2 reviewers, with the third reviewer resolving disagreements. Reviewers (M.D.S., A.L.S., T.C.H.) extracted data independently, each reviewing 8 of the following groupings: study ID, title, authorship, corresponding author appointment and contact details, country, if the study was multi-institutional, primary institution, study aims, study design, date published, study funding sources, specialty of interest, study inclusion and exclusion criteria, total participants/applications used, type of AI model, model performance factors, model endpoints, model deployment, if model is currently in use, mention of bias, bias explicitly modeled, narrative data model used, comparison to human ranking, and interpretable associations noted. Bias for this study was defined as either acknowledging or evaluating, directly or indirectly, systemic differences between protected or demographic groups (race, gender, etc). Results were summarized as data extraction fields and a selected cross-tabulation analysis was performed (Table 1).

Table 1.

Characteristics of Examined Full-Text Articles

Study Characteristics	Full-Text Articles (N=12), n (%)
Years published
<2000	1 (8)
2000-2010	1 (8)
2011-2020	0 (0)
2021-2023	10 (83)
Location
United States	12 (100)
Institution type
Academic medical centers	12 (100)
Multi-institutional	0 (0)
Study design
Cohort	6 (50)
Cross-sectional	5 (42)
Nonrandomized experimental study	1 (8)
Primary outcome
Creation of predicted rank list	9 (75)
LOR evaluation	3 (25)
Model development
Reported model performance	6 (50)
Compared to human output	6 (50)
Listed attributable features	6 (50)
Bias
Bias included in text	9 (75)
Only included as limitation	6 (42)
Bias explicitly modeled	3 (25)

Abbreviation: LOR, letters of recommendation.

Results

Search Results

We identified 1048 articles through the systematic database search. Following the removal of duplicates, 816 (78%) records remained and underwent screening. After this review, 799 (98%) records were excluded during the title and abstract screen, and the remaining 17 (2%) underwent a full-text screen for eligibility. Of these 17 articles, 5 (29%) were excluded for reasons highlighted in Figure 1, and therefore 12 (71%) were included in the scoping review.

Figure 1.

PRISMA Flow Diagram

Abbreviations: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; AI, artificial intelligence.

Study Characteristics

Most articles (10 of 12, 83%) identified for this review were published in the last 3 years, with 6 (50%) of them having been published in 2023. All 12 articles were from the United States and were published by large academic health systems with residency or fellowship programs. However, none of the articles reviewed were multi-institutional. A range of specialties were represented in the literature, including internal medicine, general surgery, internal medicine/pediatrics, neurosurgery, emergency medicine, psychiatry, and pediatric otolaryngology. Within the reviewed literature, investigators used AI to complete one of 2 tasks. They either trained AI to analyze application material to predict outcomes (9 of 12, 75%) or used it to conduct textual/sentiment analysis on the letters of recommendation (3 of 12, 25%). An overview of study characteristics and findings can be found in Table 1.

Model Development

Authors used either free text or discrete data fields from submitted applications to develop their AI model. Some models (3 of 12, 25%) attempted to integrate both data types to represent a “holistic AI review.” Models utilizing free text often worked to transform the text into discrete fields (eg, hobbies, text snippets). While half of the articles reviewed (6 of 12, 50%) did not include summary statistics of model performance, the articles that did report on model performance had only moderate average precision (Table 2). Only half of the models developed (6 of 12, 50%) compared model output to an analogous human generated output. Of the 12 articles identified, 6 reported feature importance, revealing the weights of which features are most important for the outcome variable of interest (Table 3).

Table 2.

Summary of Reviewed Articles

Publication	Model Type	Input	Output	Model Performance						Brief Description
				Sensitivitya	Specificity	AUROC	PR Curve	Accuracy	Precisionb
St John et al (2022)25	Machine learning/ natural language processing	PS	Interview invite Rank list							ML and NLP extracted metrics from PS, and regression analysis correlated output with interview offer and final rank position. ML/NLP can identify meaningful variables that differentiate candidates. Proposed use: identify applicants who would be a good fit for training programs, which can result in a more holistic and accurate match. Bias not mentioned.
Stedman et al (2009)24	LIWC	LoR Gender	Positive and negative attribute frequency							LIWC scanned LoRs to produce a quantitative profile for 16 categories. Analysis demonstrated minimal variance between LoRs. Authors question the utility of LoRs as presently structured. Possible changes are discussed. Bias incorporated into model.
Sarraf et al (2021)30	IBM Watson Natural Language Understanding Tone Analyzer	LoR	Word frequency Word sentiment							AI and computer-based algorithms reviewed the LoRs of previously matched applicants and detected linguistic differences and gender bias. Results persisted following stratification by clerkship grades and when analyzed by decade. Bias incorporated into model.
Rees et al (2023)3	Random forest	Demographics School Details Applicant Achievement (test scores, awards, grades, publications) Hobbies/interests	Rank list Matriculation			Model 1: 0.93 Model 2: 0.60	Model 1: 0.65 Model 2: 0.11			A random Forest algorithm assessed 72 variables to predict ranked applicants. The produced rank list was compared to actual outcome and demonstrated impressive accuracy. Methods were repeated to predict ranked matriculants, which demonstrated modest but better-than-random accuracy. Bias incorporated into discussion and limitations but not modeled. Model 1: Model of ranked vs unranked candidates Model 2: Model of ranked matriculants vs nonmatriculants
Pilon et al (1997)23	Artificial neural network	PS SCG/USMLE LoR Interview score	Rank list					Model 1: CC: 0.74 R2: 59.0% Model 2: CC: 0.77 R2: 59.4%		Using linear regression and then a neural network, 2 rank lists were produced. Both outputs were compared to the actual rank order submitted to the NRMP. When compared to the NRMP rank order, the 2 forecasting models performed equally well. Bias not mentioned. Model 1: Linear regression Model 2: Neural network
Ortiz et al (2023)28	Python NLTK LASSO Demographics model	Demographics ERAS features (gender, IMG, AOA, GHHS, location, PhD, no. of publications, activities) LoR	Matched applicants			Model 1: 0.75 Model 2: 0.80 Model 3: 0.72				NLORs and demographic data similarly discriminate whether applicants will or will not match into their neurosurgical residency program. NLORs potentially provide further insight regarding applicant fit. As NLORs are predictive of both Match outcomes and SLOR rankings, continuing to include narrative evaluations may be invaluable to the Match process. Bias not mentioned. Model 1: NLOR model to predict Match Model 2: Demographics model (ERAS features) Model 3: NLOR model to predict SLOR ranking
Mahtani et al (2023)15	LASSO “Bag-of-words” with SpaCy v3.0 TF-IDF	Demographics ERAS features (see Burk-Rafel et al) Notable experience Work experience Volunteer experience Research experience	Interview invite			Model 1: 0.80 Model 2: 0.92 Model 3: 0.92	Model 1: 0.49 Model 2: 0.74 Model 3: 0.73			NLP-based artificial intelligence tool to promote holistic residency application review. Proposed use: identify applicants screened out using traditional metrics. Bias acknowledged as a limitation only. Model 1: NLP model of narrative experience Model 2: ERAS structured data only Model 3: Combined Model 1 + Model 2
Kibble et al (2023)27	MonkeyLearn Lexalytics MeaningCloud	MSPE	Rank list							The rubric for manual grading provided reliable interfaculty scoring and ranking of MSPEs. While the MLMs accurately detected positive sentiment in the MSPEs, they were unable to provide reliable rank ordering compared to human raters. Bias indirectly acknowledged in discussion.
Gray et al (2022)29	VADER	Demographics LoR	Polarity score							Bias in LORs, as reflected as differences in polarity, is likely a result of the intensity of the emotions being used and not the types of emotions being expressed. NLP shows promise in identification of subtle areas of bias that may influence an individual’s likelihood of successful matching. Bias incorporated into model.
Drum et al (2023)26	RoBERTa	PS LoR MSPE Notable experience	Interview Invite	0.62	0.97				0.64	MLM created that can identify several values important for resident success in internal medicine–pediatrics programs with moderate sensitivity and high specificity using text snippets. Bias acknowledged as a limitation only.
Chillakuru et al (2022)22	BERT PCA XGBoost	PS No. of publications No. of presentations ENT residency Doximity rating USMLE scores Gender	Applicant fit Post-fellowship achievement	Model 1: 0.00 Model 2: 0.20 Model 3: 0.20		Model 1: 0.42 Model 2: 0.76 Model 3: 0.75		Model 1: 0.71 Model 2: 0.76 Model 3: 0.81	Model 1: 0.00 Model 2: 0.50 Model 3: 1.00	Demonstrated ability for document embeddings to capture meaningful information in personal statements from pediatric otolaryngology fellowship applicants. Bias acknowledged as a limitation only. Model 1: Applicant characteristics and PS cluster Model 2: raw BERT vector representation of PS Model 3: raw BERT vector and applicant characteristics
Burk-Rafel et al (2021)21	Random forest LightGBM XGBoost	Demographics ERAS features (USMLE result, awards, LoR, experiences [count/hours], medical school)	Interview invite	0.91	0.85	Model 1: 0.95 Model 2: 0.94 Model 3: 0.93	Model 1: 0.76 Model 2: 0.72 Model 3: 0.76			Developed MLM that assessed 61 ERAS features to predict probability of interview invite. Integrated MLM into interactive decision strategy tool for second pass review of applications previously excluded by human reviewers. Bias incorporated into discussion and limitations but not modeled. Model 1: All variables included Model 2: USMLE scores excluded Model 3: Incomplete applications excluded

^aRecall and Sensitivity are synonymous in literature.

^bPrecision and PPV are synonymous in literature.

Abbreviations: AUROC, area under the receiver operating characteristic; PR, precision-recall; PS, personal statement; MLM/ML, machine learning model; NLP, natural language processing; LIWC, linguistic inquiry word count version 2.063; LoR, letters of recommendation; AI, artificial intelligence; SCG, scores and clerkship grades; USMLE, United States Medical Licensing Examination; CC, correlation coefficient; NRMP, National Resident Matching Program; NLTK, Natural Language Toolkit; ERAS, Electronic Residency Application Service; IMG, international medical graduate; AOA, Alpha Omega Alpha; GHHS, Gold Humanism Honor Society; NLOR, narrative letter of recommendation; SLOR, standardized letter of recommendation; TF-IDF, Term Frequency-Inverse Document Frequency; MSPE, Medical Student Performance Evaluation; ENT, ear, nose, and throat.

Table 3.

Model Feature Importance for Articles Creating Rank List (N=9)

Author	Year	Title	Country	Institution	Clinical Area	Outcome	Features Explicitly Mentioned
St John, et al25	2022	Match Maker: Assessing Applicant Personal Statements With Artificial Intelligence	United States	University of Maryland Medical Center	General surgery	Interview invite Rank list
Rees, et al3	2023	Machine Learning for the Prediction of Ranked Applicants and Matriculants to an Internal Medicine Residency Program	United States	Geisel School of Medicine at Dartmouth	Internal medicine	Rank list Matriculation	Ranked applicants: medical school type, medical school state, medical school country, medical degree type, USMLE Step 2 CK Ranked matriculants: medical school country, contact address state, permanent address state, medical school type, hobbies (creative writing)
Pilon, et al23	1997	Neural Network and Linear Regression Models in Residency Selection	United States	University of New Mexico School of Medicine	Emergency medicine	Rank list	Medical school grades, written autobiography, interviews, letters of recommendation, and part one of the National Board scores
Ortiz, et al28	2023	Words Matter: Using Natural Language Processing to Predict Neurosurgical Residency Match Outcomes	United States	Vanderbilt University Medical Center	Neurosurgery	Matched applicants	AOA, USMLE Step 1 score, no. of publications, PhD, no. of volunteer activities, no. of research activities, gender, Gold Humanism Honor Society member, no. of work activities, current resident status, international medical graduate
Mahtani, et al15	2023	A New Tool for Holistic Residency Application Review: Using Natural Language Processing of Applicant Experiences to Predict Interview Invitation	United States	New York University Grossman School of Medicine	Internal medicine	Interview invite	Phrases indicating active leadership, research, or work in social justice and health disparities were associated with interview invitation.
Kibble, et al27	2023	Comparing Machine Learning Models and Human Raters When Ranking Medical Student Performance Evaluations	United States	University of Central Florida College of Medicine (UCF)	Medical students at UCF	Rank list
Drum, et al26	2023	Using Natural Language Processing and Machine Learning to Identify Internal Medicine–Pediatrics Residency Values in Applications	United States	University of Utah School of Medicine	Internal medicine– pediatrics	Interview invite	Academic strength, communication, compassion, DEI, leadership, self-awareness, teamwork, work ethic
Chillakuru, et al22	2022	Deep Learning for Predictive Analysis of Pediatric Otolaryngology Personal Statements: A Pilot Study	United States	Children’s National Health System	Pediatric otolaryngology fellowship	Applicant fit Post-fellowship achievement	Doximity residency ranking was the most significant multivariate regression variable for post-fellowship research output. Did not discuss feature importance other than personal statements.
Burk-Rafel, et al21	2021	Development and Validation of a Machine Learning–Based Decision Support Tool for Residency Applicant Screening and Review	United States	New York University Grossman School of Medicine	Internal medicine	Interview invite	Invite: school research rank, non-White/non-Asian race, Northeast area school, Gold Humanism Honor Society member, USMLE Step 1 Do not invite: school research rank, USMLE Step 1, USMLE Step 2 CK, internal medicine recommendation letter count, Northeast area school

Abbreviations: USMLE, United States Medical Licensing Examination; CK, clinical knowledge; AOA, Alpha Omega Alpha; DEI, diversity, equity, and inclusion; MSPE, Medical Student Performance Evaluation.

Acknowledgment of Bias

Only a small proportion of reviewed articles explicitly address bias in their final models (3 of 12, 25%). In the remaining articles, 6 articles (50%) listed bias as a limitation without explicitly modeling bias, while the last 3 articles (25%) did not discuss bias.

Discussion

This scoping review aimed to assess how AI has been developed for residency or fellowship application evaluation and how model bias has been explored within the research literature. Our review identified a limited body of literature, with only 12 studies meeting inclusion criteria. These studies primarily focused on 2 AI use cases in academic residency and fellowship programs: (1) predicting rank lists from application features and (2) conducting textual analysis such as summarizing personal statements or letters of recommendation. Although these technologies hold promise for improving the residency selection process, few studies directly address bias.

While AI tools are still in the early stages of development, their use in evaluating residency and fellowship applications could introduce bias if not carefully designed.¹⁶ Only a few studies in our review considered bias as a part of their methodology, a concerning omission given that AI models trained on skewed historical training data can perpetuate harmful patterns. Furthermore, there is no evidence that current AI models can create an output akin to that produced by a program’s current methods. The lack of inter-reliability and comparison between AI and human methods prevents conclusions about AI’s ability to replicate or improve decision-making. This is especially true when explainability measures, like feature importance, remain absent from published analyses.

The variability in AI model development and validation also highlights the need for unified guidelines and reporting standards.³³ Half of the studies we reviewed included performance summary statistics. However, what was reported varied, meaning there is still a pressing need for universally accepted reporting guidelines. Other academic domains have published standards, and these guidelines can act as a guide for AI data reporting in medical education.^34-37 We advocate for the inclusion of accuracy, sensitivity, specificity, precision, feature importance, explicit bias and fairness assessments, program disclosure, and exploring potential implications of model implementation in future publications.³⁸ Without such standards, accepting black box models where internal processes are not observable poses risks to transparency and fairness, potentially hindering the positive benefits of diversity in residency programs.^39,40

Explainable AI, which can clarify how specific aspects of applications are weighted in decision-making, holds promise for improving equity and transparency. One potential workflow of explainable AI is shown in Figure 2. However, transparency also presents the risk that applicants might tailor their materials to game the system.^41-43 Nevertheless, AI offers the potential for more consistent, transparent, and equitable application review. Further, feature importance can provide internal transparency to confirm emphasis on mission-driven values, such as research experience or clinical interest. It should be noted that bias against protected categories, such as race or gender, should be carefully mitigated. It is crucial to address algorithmic bias through explainability tools, such as SHAP (SHapley Additive exPlanations) values and LIME (Local Interpretable Model-agnostic Explanations), and to incorporate fairness audits into model development, such as IBM’s AI Fairness 360 and the University of Chicago’s Aequitas Bias and Fairness Audit Toolkit.^37,44-46

Figure 2.

Potential Use of Artificial Intelligence in Residency Application Review

Abbreviation: AI, artificial intelligence.

Note: Once applications are received, holistic review can establish applications that the program would like to interview. Within holistic review, AI review can play a part in evaluating and monitoring biases that occur during the holistic review process compared to predetermined aspirational residency attributes. Note, in this proposed framework, AI does not create a de novo rank list without human oversight.

Despite the limitations of our study, including that the referenced articles did not have a single standard method of model evaluation (limiting the comparisons of performance), the use of a single reviewer for our initial screening of titles and abstracts rather than 2, and the exclusion of gray literature, this scoping review offers a comprehensive assessment of AI use in residency and fellowship recruitment and the explicit assessment of model bias. It is also important to note that while AI has been widely used in other fields for screening applications^18,47,48 and has shown both benefits and risk,^49-57 the residency selection process remains unique, and conclusions from other sectors may not be fully transferable.

Conclusions

AI is being developed for residency and fellowship application evaluation, with current applications including applicant outcome prediction and textual analysis. Our review reveals a limited body of literature on these methods, with insufficient exploration of bias and fairness in AI models. Review of the literature identified gaps in standardized reporting of model performance, including metrics such as feature importance and explainability.

Editor’s Note

The online supplementary data contains the search strategies used in the study and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews checklist.

Author Notes

Funding: The authors report no external funding source for this study.

Conflict of interest: The authors declare they have no competing interests.

This work was previously presented at the Duke University AHEAD Education Conference, March 22, 2024, Durham, North Carolina, USA.