Abstract
Objective
We explored the impact that attributes of US medical school seniors have on their success in matching to a surgical residency, in order to analyze trends for National Resident Matching Program (NRMP) match outcomes in surgical specialties between 2007 and 2009.
Methods
Using Electronic Residency Application Service data and NRMP outcomes, we analyzed medical students' attributes and their effect in successfully matching students into residency positions in surgery, otolaryngology, orthopedic surgery, plastic surgery, and obstetrics and gynecology. Attributes analyzed included self-reported United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores, Alpha Omega Alpha (AOA) Honor Medical Society membership, research experience, additional graduate degree status, and graduation from a top 40 National Institutes of Health (NIH)–funded medical school. Odds ratios were calculated for each criterion, and 95% confidence intervals were used to determine significance.
Results
Between 2007 and 2009, the number of surgical specialty residency positions increased by 86, and the number of applicants increased by 34. Membership in AOA, USMLE Step 1 and Step 2 scores, research experience, and graduation from a top 40 NIH-funded medical school frequently had a significant impact on residents successfully matching into many specialties, while additional graduate degrees had no effect on matching into surgical specialties (range 0.64 to 1.2).
Conclusions
Although the statistical significance varied across specialties, higher USMLE Step 1 and Step 2 scores, AOA membership, research experience, and graduation from a top 40 NIH-funded medical school generally had a positive impact on match success to surgical residency for US allopathic seniors. Test preparation and seeking research experience during undergraduate medical education may be effective approaches for increasing the likelihood of success for US seniors matching into a surgical specialty.
Introduction
Residency is a critical step in the education of a physician. The National Resident Matching Program (NRMP) seeks to optimize the outcome for programs and applicants, but matching into a residency program is a competitive process of selection by both applicants and program directors. Applicants and their medical school advisors would be grateful for information that could increase applicants' potential to successfully match.
In response to queries about how applicant qualifications affected match success, the Association of American Medical Colleges (AAMC)1 published the report “Charting Outcomes in the Match” in 2006. In 2007 and 2009, the AAMC added new specialties and several new outcomes and analyses to predict match success. These reports are available online.1–3 In this study, we analyzed data from the match reports for 2007 and 2009 to identify the components of residency applications that appear to have a substantial impact on US allopathic seniors' success in matching into a surgical residency, and to determine trends over the 2-year period. Surgical specialties analyzed comprise surgery, plastic surgery, orthopedic surgery, otolaryngology, and obstetrics and gynecology.
Methods
Data for these 5 surgical specialties were extracted from the 2007 and 2009 “Charting Outcomes in the Match” reports.2,3 Permission was obtained from the AAMC. A successful match was defined as one in which applicants matched to the first-ranked specialty on their match list independent of the rank order (ie, success was measured in terms of matching to any program in their specialty of choice). An unsuccessful match was one in which an applicant either matched to another specialty or remained unmatched. In 2007 there were 3181 surgical residency positions offered, with match results available for 3676 fourth-year US medical students who sought to match into a surgical specialty; in 2009 there were 3267 positions available and 3710 applicants who submitted rank lists for a surgical specialty (table 1).
Table 1.
Total Number of Surgical Residency Positions and Applicants for 2007 and 2009
Data
We analyzed United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores, Alpha Omega Alpha (AOA) Honor Medical Society membership, research experience, whether the applicant had or was about to receive an additional graduate degree, and whether the applicant graduated from a top 40 National Institutes of Health (NIH)–funded medical school. “Research experience” was defined as any abstract, presentation, or publication. Additionally, PhD applicants and other graduate degree applicants were combined into a single category labeled “additional graduate degree.” The AAMC data include graduate degrees earned prior to medical school that were reported to the American Medical College Application Service and degrees obtained during medical school, which are tracked in the AAMC student record.1 For the purpose of our analysis, we divided USMLE Step 1 and Step 2 scores into 3 groups: less than or equal to 210, 211 to 230, and greater than 230. All calculations involving USMLE scores are in relation to the lowest group of scores (≤210). This was to facilitate a comparison among specialties, since each specialty has a different mean score for individuals that are accepted in the match.
Statistical Analysis
We calculated odds ratios for each specialty and each criterion. Odds ratios provide an estimate of the relative increase in the odds of successfully matching that can be attributed to specific criteria when other criteria are constant. Ninety-five percent confidence intervals were used to determine significance.
Results
Table 1 shows the total number of positions and applicants in 2007 and 2009. The ratio of applicants to positions was 1.2 and 1.1 for 2007 and 2009, respectively. Average USMLE Step 1 scores for matched and unmatched applicants are found in Table 2, and average USMLE Step 2 scores for matched and unmatched applicants are found in Table 3. Table 4 shows the odds ratios for each criterion reviewed for each individual specialty. Figures 1 and 2 are line graphs representing the percentage of applicants matched for their reported USMLE Step 1 score.
Table 2.
Mean United States Medical Licensing Examination Step 1 Scores of Applicants that Matched or Did Not Match Into a Given Specialty
Table 3.
Mean United States Medical Licensing Examination Step 2 Scores of Applicants that Matched or Did Not Match Into a Given Specialty
Table 4.
Odds Ratios for Reviewed Criteria
Figure 1.
Percentage of Applicants Matched Versus Mean Step 1 United States Medical Licensing Examination Score in 2007
Figure 2.
Percentage of Applicants Matched Versus Mean Step 1 United States Medical Licensing Examination Score in 2009
Odds ratios for AOA membership in 2007 were as follows: general surgery, 6.2; plastic surgery, 10.1; orthopedic surgery, 5.7; otolaryngology, 6.5; and obstetrics and gynecology, 13.4. In 2009, the odds ratios for general surgery and orthopedic surgery increased (9.1 and 9.8, respectively) while the odds ratios for plastic surgery, otolaryngology, and obstetrics and gynecology decreased (4.4, 5.9, and 10.6, respectively). In both years and for all specialties, membership in AOA had a statistically significant positive impact on successfully matching.
Higher USMLE Step 1 scores also have a positive impact on matching. The odds of matching increased dramatically over both years with scores above 230. In both years, the odds of matching reached statistical significance with Step 1 scores above 230. In 2007, the most dramatic result was found in plastic surgery with an odds ratio of 24.1. In 2009, scores above 230 had the highest impact in otolaryngology (odds ratio 17.7).
Higher USMLE Step 2 scores did not always significantly improve the odds of matching. In both years, scores greater than 211 on USMLE Step 2 significantly improved the odds of matching into general surgery, orthopedic surgery, and obstetrics and gynecology. Otolaryngology applicants had a significant increase in odds of matching for scores greater than 230 in both years. The significance of Step 2 scores in plastic surgery seems less certain with scores of 230 and above failing to reach statistical significance for matching.
In 2007, research experience was insignificant for a successful match to all specialties except for orthopedic surgery. Orthopedic surgery had a statistically significant odds ratio of 1.8. In 2009, research experience seemed to play a more significant factor in a successful match; all fields except otolaryngology had statistically significant odds ratios for this data point (table 4).
Graduation from a top 40 NIH-funded medical school played a significant role in matching to general surgery, orthopedic surgery, and obstetrics and gynecology in 2007. In 2009, this criterion also significantly impacted match success in otolaryngology. Medical school designation by the NIH as one of the top 40 funded schools had no impact on matching to plastic surgery.
Additional medical degrees, including PhDs, had no significant effect on successfully matching into any specialty in either year. Odds ratios ranged from 0.6 to 1.2.
Discussion
The demand for surgical residencies currently exceeds the number of positions offered (table 1). Plastic surgery offered the fewest positions and had the highest ratio of applicants to positions for both 2007 and 2009. Obstetrics and gynecology had the highest number of applicants each year. General surgery had the lowest applicant-to-position ratio for both years.
USMLE Scores
Frequently the first criterion reviewed by residency programs is the USMLE Step 1 score. These scores are useful for residency programs in their initial screening of applicants because they are one of the only tools for objective, standardized evaluation across all medical schools. A successfully matched candidate into plastic surgery had a score of 241 and 245, followed closely by otolaryngology with 238 and 240, for 2007 and 2009, respectively. Those scores typically fall above the 80th percentile. Average scores for unsuccessful candidates for plastic surgery and otolaryngology were still above the 55th percentile. To further demonstrate this point, approximately 80% to 85% of applicants who scored between 201 and 210 matched successfully into a general surgery residency, whereas fewer then 20% of applicants that scored in the same range matched successfully into a plastic surgery residency. To have a greater than 90% chance of matching to general surgery, an applicant needed a score of at least 210 in 2007 and 222 in 2009. For plastic surgery, this meant a score of at least 240 in 2007 and 260 in 2009. Higher USMLE Step 1 scores correlated to higher rates of match success into a given surgical specialty.
The same holds true for USMLE Step 2 scores, the higher an applicant scored, the better the chance of matching. Not all applicants have their Step 2 results before they interview, but if the applicant has a score above 230, it has a clear positive impact. A score above 230 increased the odds ratio of a successful match between 8- and 35-fold across all specialties. Plastic surgery required the highest scores.
AOA Membership
Membership in the Alpha Omega Alpha Honor Medical Society4 can significantly strengthen a residency application. Often inclusion in AOA is a primary factor reviewed by a residency program in a competitive specialty. Approximately halfway through a medical student's third year of school, faculty and administration are asked to identify the top students in the class. Initially the top sixth of students, measured by academic performance, are selected for consideration for AOA membership. Later, that group of students is further narrowed on the basis of academic performance and other attributes such as leadership, professionalism, and commitment to service; these students become AOA members. Certain schools do not participate in AOA or have AOA selection after the residency match is complete, which complicates interpretation of this data.
Regardless of surgical specialty, AOA members have higher rates of acceptance into their respective programs. However, the impact of membership on success rates is clearly variable between specialties. In both 2007 and 2009, AOA membership had the largest effect on successfully matching into obstetrics and gynecology, which is shown by an acceptance rate increase from 87% to 99% when comparing non-AOA members and AOA members. Membership in AOA increased the likelihood of successfully matching into general surgery, orthopedic surgery, otolaryngology, and obstetrics and gynecology by more than 6 times in 2007 and 2009. While USMLE scores are often a great indicator of how competitive an applicant is for a given residency program, AOA membership can solidify that applicant's chances of matching successfully.
Research Experience
Research experience enhances an applicant's probability of matching into a surgical specialty. The presence of research experience improved the odds of successfully matching across all surgical specialties, with an odds ratio of at least 1.1 in both matches. The difference in acceptance rates in 2007 was most marked in plastic surgery, as seen by the 14.5% increase in rates between applicants with no research experience and those with any research experience. The acceptance rate based on research experience jumped substantially in plastic surgery in 2009 with an increase of 38%. Though the increased rate of success is not as dramatic as it is for some other variables, the value of research experience for an applicant is still worth noting.
Additional Graduate Degrees
Additional graduate degrees have very little impact on successful matching. In both years, the odds ratio ranged from 0.6 to 1.2. This did not reach statistical significance.
Medical School
A student's attendance at a top 40 NIH-funded medical school appeared to add a small advantage to applicants. Applicants to otolaryngology have the highest odds ratio compared to other surgical specialties with an odds ratio of 2.1 and 2.5 in 2007 and 2009, respectively. Plastic surgery seems to place the least importance on top 40 NIH-funded medical schools, with an odds ratio of 1.3 and 1.2 in 2007 and 2009, respectively.
Limitations
Our study has several limitations. One is its reliance on self-reported data. For example, a candidate's research contributions are not verified by the Electronic Residency Application Service or the NRMP. There is evidence5 that the research listed by at least a small subset of applicants may in fact be “ghost” research or research that could not be verified to have been completed. The same could be true of AOA status and graduate degrees, although this is admittedly unlikely due to the severe repercussions of such behavior. Our paper also fails to evaluate other data that may be important, such as class ranking and honors credits. Another limitation is that the 2008 data is not available from the AAMC. Trends may have been easier to follow if that data were available.
Residency program directors are looking for the best candidates for their program, considering all aspects of a student's application and the interview. A limitation of the AAMC data and this report is that neither takes into account intangible factors such as away rotations and personal interactions. High USMLE scores, AOA membership, and research experience influence the chance of matching but do not guarantee success. The interview plays a role in a successful match but was not assessed in the context of this paper.
Conclusion
Our paper provides objective and valuable information to both advisors in and applicants to these competitive fields. Higher USMLE Step 1 and Step 2 scores, AOA membership, research experience, and graduation from a top 40 NIH-funded medical school all had a positive impact on medical students' match success into a surgical residency. Additional graduate degrees had no impact on matching. Future studies are needed to examine whether increasing the number of surgical residency positions would dilute the quality of successful candidates or whether there is an ideal applicant-to-position ratio that would allow surgical residency coordinators to remain selective in their choices.
Footnotes
Both authors are at Scott & White Healthcare and Texas A&M Health Science Center. Jeremy R. Rinard, MD, is Chief Resident in the Division of Plastic Surgery, Department of Surgery; and Raman C. Mahabir, MD, FRCSC, is Associate Professor, Assistant Program Director, and Chief of Microsurgery in the Division of Plastic Surgery, Department of Surgery.
The authors have no conflicts of interest to disclose.
References
- 1.Association of American Medical Colleges and National Resident Matching Program. Charting outcomes in the match (2006) Available at: http://www.nrmp.org/data/matchoutcomes2006.pdf. Accessed December 29, 2009.
- 2.Association of American Medical Colleges and National Resident Matching Program. Charting outcomes in the match, 2nd Edition (2007) Available at: http://www.nrmp.org/data/chartingoutcomes2007.pdf. Accessed December 29, 2009.
- 3.Association of American Medical Colleges and National Resident Matching Program. Charting outcomes in the match, 3rd Edition (2009) Available at: http://www.nrmp.org/data/chartingoutcomes2009v3.pdf. Accessed December 29, 2009.
- 4.Alpha Omega Alpha Honor Medical Society. Constitution (ratified 2009) Available at: http://www.alphaomegaalpha.org/constitution.html. Accessed December 29, 2009.
- 5.Kuo P. C., Schroeder R. A., Shah A., Shah J., Jacobs D. O., Pietrobon R. “Ghost” publications among applicants to a general surgery residency program. J Am Coll Surg. 2008;207(4):485–489. doi: 10.1016/j.jamcollsurg.2008.04.029. [DOI] [PubMed] [Google Scholar]