Abstract
Objective
The primary objective was to describe emergency medicine (EM) residency selection criteria.
Methods
A survey was sent to the Council of Emergency Medicine Residency Directors listserv. Respondents were asked to rank order the various components of the application on a Likert scale from 1 (minimally important) to 10 (highly important). The mean ranking and standard deviation for each of the components were calculated. The survey sought to determine characteristics associated with offering an applicant an invitation to interview and subsequent ranking. Percentages with defined minimum requirements were calculated. Comparisons across residency length and location were completed with a Pearson chi‐square test for categorical variables and Student's t‐test for continuous variables.
Results
A total of 120 surveys were completed. The highest ranked components included away/visiting institution departmental standardized letter of evaluation (SLOE) (mean ± SD = 8.80 ± 1.25), residency interview (mean ± SD = 8.74 ± 1.28), home institution departmental SLOE (mean ± SD = 8.61 ± 1.18), away/visiting institution EM rotation grade (mean ± SD = 8.29 ± 1.43), and home institution EM rotation grade (mean ± SD = 8.07 ± 1.42). The most consistently ranked items included home institution departmental SLOE (SD = 1.18), away/visiting institution departmental SLOE (1.25), and residency interview (1.28). Characteristics associated with offering an interview to an applicant included only 10% of responses indicating a United States Medical Licensing Examination Step 1 score of 220 was needed. At least one SLOE was required in 80% of responses. Program location was related to the number of SLOEs required (p = 0.03). Length of residency and program location differed significantly in how a residency ranked components when considering an applicant (p < 0.05).
Conclusion
Emergency medicine programs put high value in departmental SLOEs, the interview, and EM rotation grades when selecting potential residents. Higher value is placed on SLOEs and grades from away/visiting institutions compared with students’ home institutions.
Emergency medicine (EM) residency positions are currently in high demand. With increasing numbers of medical school graduates applying for this specialty,1 the competition for available spots is on the rise. To provide each student with the best chance of matching into his or her desired specialty and program, it is important to decrease the ambiguity among medical students and advisors concerning which items should be prioritized when building their application. Likewise, students should have an accurate picture of their competitiveness as an applicant. The last survey to address the components of the National Resident Matching Program (NRMP) application that were most important to EM program directors in their consideration of an applicant was conducted in 1998.2 EM residencies have undergone significant changes since that time. In 1998, there were 934 EM PGY‐1 spots in the United States representing 4.6% of all PGY‐1 spots with 94% of spots filling.2 In contrast, these numbers rose to 1,895 EM PGY‐1 positions in 2016, representing 6.8% of all PGY‐1 positions with 99.9% of EM PGY‐1 spots filling.1 Additionally, the number of residency applications has increased markedly. In 2012, there were a total of 100,320 applications received by residency programs. That number has increased to 158,360 in 2016.3
As EM residency programs become more competitive and receive an increasing number of medical student applications, they must more clearly define the aspects of a successful application. For example, they may have objective measures such as a minimum United States Medical Licensing Examination (USMLE) Step 1 score or number of standardized letters of evaluation (SLOEs) to screen applicants for an interview. It is possible that attributes in the NRMP application are weighted differently than previously reported in the 1998 study to meet current demands. Programs may now also have the freedom to require that USMLE Step 2 clinical knowledge (CK) and/or clinical skills (CS) scores be disclosed before ranking applicants. With the broad range of EM programs, it is possible that factors such as location, length, and type of training environment contribute to different requirements by different programs. This study seeks to examine the relative emphasis EM residency programs place on different aspects of the NRMP application, as well as determining the characteristics of medical students likely to obtain residency interviews.
Methods
Study Design
Data were obtained via a voluntary, anonymous, electronic survey at an academic, tertiary care, Level I trauma center and home of an EM residency program. The survey and protocol were reviewed by the local institutional review board and deemed exempt.
Selection of Participants
A recruitment e‐mail containing an electronic link to the survey was sent to the Council of Emergency Medicine Residency Program Directors (CORD) listserv. This listserv consists of program directors, associate/assistant program directors, and other educational leadership from EM residencies throughout the United States. The recruitment e‐mail asked that only one individual from each institution complete the survey.
Protocol and Measurements
The survey was developed through a Qualtrics program using similar methods as Crane and Ferraro;2 however, participants were asked to use a 10‐point Likert scale (1 = least important, 10 = very important), rather than a 5‐point scale to assign value to EM rotation grades and departmental SLOEs from a student's home institution, EM rotation grades and departmentally written SLOEs obtained on visiting or away rotations, individually written SLOEs, the residency interview, USMLE Step 1 and 2 CK and CS scores, grades in other clinical clerkships, preclinical/basic science grades, the Dean's letter, membership in Alpha Omega Alpha (ΑΩΑ), extracurricular/volunteer work, personal statement, and research/publications (Table 1). In addition, survey questions included what minimum USMLE Step 1 and Step 2 CK scores were necessary to offer an interview invitation to an applicant and to subsequently rank that applicant, if there were a certain number of SLOEs required to offer an applicant an interview. and what the recommended number of away rotations is for the average applicant. Finally, the survey included a free‐text section for anonymous comments.
Table 1.
Survey Elements Ranked on Likert Scale (1–10)
| Away/visiting institution department SLOE |
| Residency interview |
| Home institution departmental SLOE |
| Away/visiting institution EM rotation grade/evaluation |
| Home institution EM rotation grade/evaluation |
| Overall clinical clerkship grades |
| USMLE Step 2 CK score |
| Individual SLOEs |
| Expressed interest in program |
| USMLE Step 1 score |
| Dean's letter |
| Extracurricular/volunteer work |
| ΑΩΑ status |
| USMLE Step 2 CS score |
| Personal statement |
| Overall basic science grades |
| Research/publications |
CK = clinical knowledge; CS = clinical skills; SLOE = standardized letter of evaluation; USMLE = United States Medical Licensing Examination.
Data Analysis
Data were compiled into mean with standard deviation (SD) for all application items ranked on the Likert scale. Responses to minimum scores for USLME Step 1 were divided into the following categories: pass, 200–210, 210–220, 220–230, and >230. Minimum USMLE Step 2 CK scores were divided into the following categories: none required, 200–210, 210–220, 220–230, and >230. Number of SLOEs required to offer an interview were divided in to the following categories: none, 1, 2, and >2. The number of recommended away rotations were divided into the following categories: none, 1, 2, and >2. Percentages for each of the categories were then calculated. Comparisons by the program location (based on the Society for Academic Emergency Medicine [SAEM] regions) and length of residency (3‐year vs. 4‐year programs) were completed using the Pearson chi‐square for categorical variables and the Student's t‐test for continuous variables. For differences in means, the mid‐Atlantic program location and 3‐year program length were chosen as the referent to make comparisons to as they had the largest number of participants.
Results
The survey was open from March to April 2016, during which time 120 responses were recorded. Demographic data included 68% of respondents were from 3‐year programs, whereas the remainder were from 4‐year programs. Primary training environments were academic (67%), county hospital–based (17%), and community (16%) programs. The locations of the participating programs were from the following SAEM regions: 29% mid‐Atlantic, followed by Great Plains (18%), Western (15%), Midwest (12%), Southeastern (10%), Southwestern (9%), and finally New England (7%).
Characteristics associated with offering an interview to an applicant and subsequently ranking an applicant are displayed in Table 2. A small minority of programs (10%) require a USMLE Step 1 score of greater than 220 to offer an applicant an interview with the majority (64%) requiring a score of 210 or less. The majority of programs (78%) do not require a USMLE Step 2 CK score to offer an interview, although most (54%) require to rank an applicant. At least one SLOE was required by 80% of programs to offer an interview, with many (38%) requiring two or more. The majority of programs (98%) recommended completing at least one away/visiting rotation, but only 31% recommended completing more than one of these. Program location was associated with required a different number of SLOEs (p = 0.03) but no differences were found by length of residency.
Table 2.
Characteristics Associated With Offering an EM Residency Interview or Ranking an Applicant
| Offering an Interview | All | Program Location | Program Length | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Great Plains | Mid‐Atlantic | Midwest | New England | Southeastern | Southwestern | Western | p‐value | 3 Years | 4 Years | p‐value | ||
| (n = 21) | (n = 33) | (n = 14) | (n = 8) | (n = 11) | (n = 10) | (n = 17) | (n = 78) | (n = 36) | ||||
| Minimum USMLE Step 1 score | ||||||||||||
| Pass | 52 (46) | 10 (47.6) | 16 (48.5) | 7 (53.9) | 3 (42.9) | 3 (27.3) | 3 (30.0) | 10 (58.8) | 0.8 | 33 (44.6) | 17 (48.6) | 0.06 |
| 200–210 | 20 (18) | 6 (28.6) | 7 (21.2) | 1 (7.8) | 0 | 3 (27.3) | 2 (20.0) | 1 (5.9) | 16 (21.6) | 4 (11.4) | ||
| 210–220 | 29 (26) | 4 (19.1) | 7 (21.2) | 4 (30.8) | 3 (42.9) | 3 (27.3) | 4 (40.0) | 4 (23.6) | 21 (28.4) | 7 (20.0) | ||
| 220–230 | 11 (10) | 1 (4.8) | 3 (9.1) | 1 (7.8) | 1 (14.3) | 2 (18.2) | 1 (10.0) | 2 (11.8) | 4 (5.4) | 7 (20.0) | ||
| >230 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||
| Minimum USMLE Step 2 CK score | 0.1 | 0.7 | ||||||||||
| No requirement | 82 (74) | 19 (95.0) | 20 (62.5) | 7 (53.9) | 6 (75.0) | 10 (90.9) | 6 (60.0) | 14 (82.4) | 52 (70.3) | 28 (80.0) | ||
| 200–210 | 4 (4) | 0 | 2 (6.3) | 2 (15.4) | 0 | 0 | 0 | 0 | 4 (5.4) | 0 | ||
| 210–220 | 12 (11) | 1 (5.0) | 6 (18.8) | 2 (15.4) | 1 (12.5) | 0 | 0 | 2 (11.8) | 9 (12.2) | 3 (8.6) | ||
| 220–230 | 11 (10) | 0 | 4 (12.5) | 2 (15.4) | 1 (12.5) | 1 (9.1) | 2 (20.0) | 1 (5.9) | 7 (9.5) | 4 (11.4) | ||
| >230 | 2 (2) | 0 | 0 | 0 | 0 | 0 | 2 (20.0) | 0 | 2 (2.7) | 0 | ||
| No. of SLOEs | 0.03 | 0.55 | ||||||||||
| No requirement | 23 (20) | 1 (4.8) | 10 (30.3) | 2 (15.4) | 3 (37.5) | 2 (18.2) | 1 (10.0) | 4 (23.5) | 15 (20.3) | 8 (22.9) | ||
| 1 | 48 (42) | 15 (71.4) | 7 (21.2) | 7 (53.9) | 5 (62.5) | 5 (45.6) | 3 (30.0) | 6 (35.3) | 31 (41.9) | 14 (40.0) | ||
| 2 | 38 (34) | 4 (19.1) | 15 (45.6) | 4 (30.8) | 0 | 3 (23.3) | 5 (50.0) | 7 (41.2) | 24 (32.4) | 13 (37.1) | ||
| >2 | 4 (4) | 1 (4.8) | 1 (3.1) | 0 | 0 | 1 (9.1) | 1 (10.0) | 0 | 4 (5.4) | 0 | ||
| Ranking an applicant | ||||||||||||
| Require an USMLE Step 2 CK score | 60 (54) | 14 (66.7) | 17 (51.5) | 6 (46.2) | 3 (42.9) | 5 (45.6) | 9 (90.0) | 6 (35.3) | 0.1 | 43 (58.1) | 15 (42.9) | 0.2 |
| Require USMLE Step 2 CS score | 40 (35) | 9 (42.9) | 11 (33.3) | 6 (46.2) | 3 (38.5) | 5 (45.6) | 2 (20.0) | 4 (23.5) | 0.7 | 28 (37.8) | 10 (28.6) | 0.5 |
Data are reported as n (%).
CK = clinical knowledge; CS = clinical skills; SLOE = standardized letter of evaluation; USMLE = United States Medical Licensing Examination.
The responses to items listed ranked on the Likert scale are presented in Tables 3 and 4. The highest ranked items included away/visiting institution departmental SLOEs, the residency interview, home institution departmental SLOEs, away/visiting institution EM rotation grades, and home institution EM rotation grades. Away/visiting institution components (SLOEs and rotation grades) were both ranked more highly than their home institution counterparts. Additionally, departmental SLOEs (both away/visiting and home institution) were both ranked more highly than individually written SLOEs. The lowest ranked components included the personal statement, overall basic science grades, and research/publications. The most consistently ranked components with the smallest SD included the home institution departmental SLOE (SD = 1.18), away/visiting institution departmental SLOE (SD = 1.25), and the residency interview (SD = 1.28). These were also the three most highly ranked components.
Table 3.
Reported Importance When Considering an Applicant for Your EM Residency
| All | Great Plains | Mid‐Atlantica | Midwest | New England | Southeastern | Southwestern | Western | |
|---|---|---|---|---|---|---|---|---|
| (n = 114) | (n = 21) | (n = 33) | (n = 14) | (n = 8) | (n = 11) | (n = 10) | (n = 17) | |
| Away/visiting institution department SLOE | 8.8 (±1.3) | 9.1 (±0.9) | 8.8 (±1.2) | 7.8 (±2.2)b | 9.5 (±0.5) | 8.9 (±1.0) | 9.3 (±0.5) | 8.6 (±1.1) |
| Residency interview | 8.7 (±1.3) | 8.7 (±1.2) | 8.8 (±1.3) | 8.4 (±1.5) | 8.8 (±1.4) | 9.7 (±0.6)b | 9.0 (±1.1) | 8.1 (±1.2) |
| Home institution departmental SLOE | 8.6 (±1.2) | 8.7 (±1.2) | 8.6 (±1.2) | 8.0 (±1.8) | 8.9 (±0.7) | 9.1 (±0.8) | 9.1 (±0.8) | 8.4 (±1.0) |
| Away/visiting institution EM rotation grade/evaluation | 8.3 (±1.4) | 7.9 (±2.1) | 8.2 (±1.3) | 8.1 (±1.4) | 8.6 (±1.3) | 8.7 (±1.2) | 8.2 (±1.5) | 8.8 (±0.9) |
| Home institution EM rotation grade/evaluation | 8.1 (±1.4) | 7.7 (±2.1) | 8.0 (±1.3) | 7.9 (±1.6) | 8.0 (±1.0) | 8.7 (±1.0) | 8.1 (±1.4) | 8.4 (±0.8) |
| Overall clinical clerkship grades | 7.3 (±1.7) | 7.2 (±1.0) | 7.5 (±2.0) | 7.2 (±1.5) | 6.4 (±2.1) | 6.7 (±1.6) | 7.0 (±1.9) | 8.0 (±1.1) |
| USMLE Step 2 CK score | 6.5 (±2.0) | 6.3 (±1.9) | 7.2 (±1.8) | 6.8 (±1.7) | 4.9 (±1.7)b | 6.5 (±2.1) | 7.2 (±1.6) | 5.5 (±2.2)b |
| Individual SLOEs | 6.5 (±2.0) | 6.2 (±2.6) | 6.4 (±2.2) | 6.5 (±2.1) | 6.3 (±1.5) | 7.0 (±1.2) | 6.7 (±2.3) | 6.6 (±1.5) |
| Expressed interest in program | 6.0 (±2.7) | 5.1 (±2.7)b | 6.9 (±2.1) | 5.5 (±2.8) | 5.6 (±2.5) | 6.9 (±2.8) | 7.3 (±2.2) | 4.4 (±2.7)b |
| USMLE Step 1 score | 6.0 (±2.1) | 6.1 (±1.6) | 6.2 (±2.2) | 5.6 (±2.1) | 5.3 (±2.6) | 6.6 (±2.3) | 5.9 (±2.0) | 5.4 (±2.1) |
| Dean's letter | 5.5 (±2.2) | 5.6 (±2.0) | 5.2 (±2.1) | 5.8 (±2.1) | 4.5 (±2.7) | 4.4 (±2.0) | 5.9 (±2.6) | 6.6 (±2.0)b |
| Extracurricular/volunteer work | 5.1 (±1.9) | 4.1 (±1.8) | 4.1 (±1.6) | 3.9 (±2.4) | 3.3 (±2.7) | 3.8 (±1.3) | 4.2 (±2.0) | 4.7 (±1.9) |
| ΑΩΑ status | 4.9 (±2.5) | 5.3 (±2.1) | 4.5 (±2.3) | 5.5 (±2.9) | 5.5 (±3.0) | 4.5 (±3.4) | 5.0 (±2.7) | 4.7 (±2.5) |
| USMLE Step 2 CS score | 4.7 (±2.1) | 4.9 (±1.6) | 5.3 (±2.4) | 4.7 (±2.1) | 3.7 (±2.5) | 4.4 (±2.2) | 4.3 (±1.6) | 3.9 (±1.8)b |
| Personal statement | 4.6 (±2.1) | 4.3 (±2.0) | 4.7 (±2.1) | 4.9 (±1.9) | 3.0 (±1.2)b | 5.3 (±2.2) | 5.8 (±2.3) | 4.2 (±2.0) |
| Research/publications | 4.1 (±1.9) | 5.2 (±2.3) | 5.1 (±1.7) | 4.5 (±2.6) | 4.1 (±2.6) | 4.5 (±1.6) | 6.0 (±0.9) | 5.5 (±1.5) |
| Overall basic science grades | 4.1 (±1.8) | 4.6 (±1.5) | 4.2 (±1.7) | 3.5 (±1.3) | 3.7 (±2.5) | 3.4 (±1.8) | 4.6 (±1.6) | 4.0 (±2.2) |
Data are reported as mean (±SD).
CK = clinical knowledge; CS = clinical skills; SLOE = standardized letter of evaluation; USMLE = United States Medical Licensing Examination.
Mid‐Atlantic is the referent group for all comparisons.
p < 0.05, table is sorted by overall ranking from highest to lowest.
Table 4.
Reported Importance by Years of EM Residency
| All | 3 Yearsa | 4 Years | |
|---|---|---|---|
| (n = 114) | (n = 78) | (n = 36) | |
| Away/visiting institution department SLOE | 8.8 (±1.3) | 8.7 (±1.1) | 8.9 (±1.5) |
| Residency interview | 8.7 (±1.3) | 8.9 (±1.2) | 8.3 (±1.3)b |
| Home institution departmental SLOE | 8.6 (±1.2) | 8.6 (±1.2) | 8.6 (±1.3) |
| Away/visiting institution EM rotation grade/evaluation | 8.3 (±1.4) | 8.2 (±1.3) | 8.6 (±1.6) |
| Home institution EM rotation grade/evaluation | 8.1 (±1.4) | 8.0 (±1.4) | 8.2 (±1.5) |
| Overall clinical clerkship grades | 7.3 (±1.7) | 6.9 (±1.7) | 8.1 (±1.3)b |
| USMLE Step 2 CK score | 6.5 (±2.0) | 6.8 (±1.8) | 5.9 (±2.2)b |
| Individual SLOEs | 6.5 (±2.0) | 6.6 (±1.9) | 6.3 (±2.3) |
| Expressed interest in program | 6.0 (±2.7) | 6.0 (±2.6) | 5.9 (±2.7) |
| USMLE Step 1 score | 6.0 (±2.1) | 6.1 (±2.0) | 5.7 (±2.2) |
| Dean's letter | 5.5 (±2.2) | 5.1 (±2.1) | 6.4 (±2.2)b |
| Extracurricular/volunteer work | 5.1 (±1.9) | 3.8 (±1.7) | 4.8 (±2.0)b |
| ΑΩΑ status | 4.9 (±2.5) | 4.9 (±2.5) | 5.0 (±2.4) |
| USMLE Step 2 CS score | 4.7 (±2.1) | 4.7 (±2.0) | 4.5 (±2.2) |
| Personal statement | 4.6 (±2.1) | 4.7 (±2.1) | 4.4 (±2.0) |
| Research/publications | 4.1 (±1.9) | 4.8 (±2.0) | 5.6 (±1.7)b |
| Overall basic science grades | 4.1 (±1.8) | 3.9 (±1.7) | 4.5 (±2.0) |
Data are reported as mean (±SD).
CK = clinical knowledge; CS = clinical skills; SLOE = standardized letter of evaluation; USMLE = United States Medical Licensing Examination.
Three‐year program is the referent group for all comparisons.
p < 0.05, table is sorted by overall ranking from highest to lowest.
List rankings differed by residency location (Table 3). Compared to mid‐Atlantic residencies, on average, Great Plains residencies ranked expressed interest in program significantly lower (5.1 vs. 6.9), Midwest residencies ranked away/visiting institution SLOE significantly lower (7.8 vs. 8.8), residencies in the New England and Western residencies region ranked USMLE Step 2 CK score significantly lower (4.9 and 5.5, respectively, vs. 6.8), Southeastern residency ranked residency interview significantly higher (9.7 vs. 8.8), and Western residencies ranked a letter from the dean significantly higher (6.6 vs. 5.2). There were no significant differences between the mid‐Atlantic and Southwestern region rankings.
Length of residency also effected rankings (Table 4). Compared to 3‐year residencies, on average, 4‐year residencies ranked residency interview (8.3 vs. 8.9) and USMLE Step 2 CK score (5.9 vs. 6.8) lower. Four‐year residencies ranked clinical clerkship grades (8.1 vs. 6.9), Dean's letter (6.4 vs. 5.1), extracurricular or volunteer work (4.8 vs. 3.8), and research or publications (5.6 vs. 4.8) higher, on average, than 3‐year residencies.
Discussion
In similar previous work by Crane and Ferarro,2 EM rotation grades were found to be the most highly ranked component of the application, followed by the residency interview, clinical grades, “other,” and letters of recommendation.2 While those results are similar to the results found in the current study, some important differences exist. In the current study, SLOEs, particularly departmental SLOEs and SLOEs from away/visiting institutions, were found to be highly and consistently ranked. The SLOE (originally termed the standardized letter of recommendation) was introduced in 1996 by CORD in an attempt to normalize the evaluation of applicants and improve the inter‐rater reliability of letters of recommendation.4 Our current findings are consistent with those described in other work, which suggested that the SLOE was the most important factor in determining which candidates would receive interview invitations.5 While it has been shown previously that SLOE writers may not consistently use the full spectrum of variables consistently and that very few SLOE writers use the lowest category,6 program leadership continue to value the SLOE highly. This was reinforced by the finding that 80% of respondents require at least one SLOE to offer an applicant an interview with 38% requiring two or more SLOEs.
A unique aspect of the current study is the separation of SLOEs and EM grades from home institutions versus away/visiting institutions. Departmental SLOEs from away institutions were the most highly ranked component of the application and were more highly ranked than departmental SLOEs from home institutions. Similarly, EM grades from away/visiting rotations were ranked more highly than EM grades from the student's home institution. While the scores for all of these components were highly ranked, it suggests that experiences on away/visiting rotations are considered more important than those at a student's home institution. This idea is further reinforced by the finding that 98% of the respondents saw doing at least one away rotation as an important characteristic in obtaining a residency interview. Overall, these results highlight the importance for medical students in doing an away/visiting rotation at an institution that has a residency program and from whom they would be able to obtain a SLOE.
Departmental SLOEs from away/visiting institutions, as well departmental SLOEs from a student's home institution, were more highly valued than any individually written SLOE. As discussed by in previous work, a group or departmental SLOE may be more objective and provide a more accurate representation of a student.6 This again highlights the importance not only of obtaining SLOEs, but also in obtaining the type of SLOE that residency leadership finds most informative.
In 2016, McCann et al.7 sought to determine medical student's perspective on which portions of the application were most important. They found that EM rotation grades, residency interview, other rotation grades, and letters of recommendation were the most important. While these results are similar to those found in our study as well as previous work by Crane and Ferraro2 the difference in the relative importance of the SLOE is notable. The order of importance of the most highly ranked components (EM rotation grades, residency interview, clinical rotation grades, “other,” letters of recommendation, and overall grades) were the same in both of the previous studies.2, 7 This suggests that medical student's perceptions may be rightly influenced by the previous work of Crane et al. In our current study, SLOEs were found to be more important. It seems likely that as the SLOE has become more universally used and that effort has been put forth to increase the reliability, the value of the SLOE has increased relative to other components of the application.
While objective test scores such as the USMLE Step 1, Step 2 CK, and Step 2 CS were not ranked as highly as other components of the application it bears note that they are still essential to an applicant's overall success. Wagoner and Suriano8 showed that USMLE Step 1 scores may be used as a screening tool for applicants. This finding was reinforced in our study by finding that 54% of respondents cited a minimum USMLE Step 1 score to obtain an interview. Fewer respondents cited minimum USMLE Step 2 scores, however.
Consistent in our study as well as in previous work2, 7 were what responders recognized as the least important parts of the application—research/publications, overall basic science grades, and the personal statement. However, these items were still given some value, reinforcing that an applicant is dynamic with multiple factors contributing to the whole application.
Our study also sought to compare characteristics associated with obtaining a residency interview between different regions of the country and programs of differing durations. There may be a perception among medical students that there are different minimum requirements to obtain interviews depending on the location of the program or the program being a 4‐year program versus a 3‐year program. Our current study found no differences in based on SAEM region or length of program. This suggests that these characteristics are consistent throughout the country and that students do not need a “stronger” application to receive interview invitations from various programs.
When examining characteristics associated with offering an interview or ranking an applicant (Table 2), the only difference that reached statistical significance was the number of SLOEs recommended based on program location. Despite these differences, the maximum percentage that required more than two SLOEs was 10% (Southwestern region). Based on these numbers, if applicants obtain two SLOEs, they would be eligible for interviews at the vast majority of programs.
Similarly, despite some statistical differences, the relative importance of the various components of the application were generally consistent regardless of program length. Four‐year programs did place higher value on clerkship grades, the Dean's letter, extracurricular activity or volunteer work, and research or publications more highly than 3‐year programs. Applicants who are interested in matching at 4‐year programs may need to put a greater emphasis on these components of the application to maximize their chances of matching successfully.
Limitations
In an attempt to maximize the amount of data, the survey was sent to the CORD listserv. This listserv includes residency leaders including program directors, associate/assistant program directors, and departmental chairs. In addition to these residency leaders, other individuals also subscribe to the listserv including clerkship directors and others involved in EM education. It was requested in the recruitment e‐mail that only a single individual from each institution involved in the applicant review and ranking process complete the survey.
However, we cannot be sure that only a single individual from each institution completed the survey. If we were to assume that each response represents a unique institution, the response rate would be 120 of 201 residency programs, or 59.7%. This may limit the generalizability of the data. The regional data may be of limited use due to the small sample size in some of the regions, which will also cause bias toward the programs that completed the survey. However, we were able to obtain a geographically diverse group of respondents. Finally, this survey used a different scale, 1 to 10 points, as opposed to the 1‐ to 5‐point scale used in previous studies.
Conclusion
Emergency medicine programs put high value in departmental standardized letter of evaluations, the interview, and emergency medicine rotation grades and evaluations when evaluating potential residents. Overall, visiting/away rotations are highly valued based both on standardized letter of evaluations and on rotation grades. This is consistent throughout the country and is not dependent on the length of the program or the primary training environment of the program. These highest valued items are similar to past studies and studies evaluating students’ perspectives, although students may be undervaluing the importance of standardized letter of evaluations.
The authors thank the participants in the study for providing us with invaluable information that will help advisors and students alike across the emergency medicine community.
AEM Education and Training 2018;2:146–153
Presented at the Society for Academic Emergency Medicine (SAEM) National Meeting, Orlando, FL, May 2017.
The authors have no relevant financial information or potential conflicts to disclose.
References
- 1. National Resident Matching Program . Results and Data: 2016 Main Residency Match. Washington, DC: National Resident Matching Program, 2016. [Google Scholar]
- 2. Crane JT, Ferraro CM. Selection criteria for emergency medicine residency applicants. Acad Emerg Med 2000;7:54–60. [DOI] [PubMed] [Google Scholar]
- 3. Marco CA, Nelson LS, Baren JM, et al. American Board of Emergency Medicine report on residency and fellowship training (2016‐2017). Ann Emerg Med 2017;69:640–52. [DOI] [PubMed] [Google Scholar]
- 4. Keim SM, Rein JA, Chisholm C, et al. A standardized letter of recommendation for residency application. Acad Emerg Med 1999;6:1141–6. [DOI] [PubMed] [Google Scholar]
- 5. Love JN, Smith J, Weizberg M, et al. Council of Emergency Medicine Residency Directors’ standardized letter of recommendation: the program director's perspective. Acad Emerg Med 2014;21:680–7. [DOI] [PubMed] [Google Scholar]
- 6. Grall KH, Hiller KM, Stoneking L. Analysis of the evaluative components on the standard letter of recommendation (SLOR) in emergency medicine. West J Emerg Med 2014;15:419–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. McCann SD, Nomura JT, Terzian WT, Breyer MJ, Davis BJ. Importance of the emergency medicine application components: the medical student perception. J Emerg Med 2016;50:466–70. [DOI] [PubMed] [Google Scholar]
- 8. Wagoner NE, Suriano R. Program director's responses to a survey on variables used to select residents in a time of change. Acad Med 1999;74:51–8. [PubMed] [Google Scholar]