Evaluation of Factors Associated With Successful Matriculation to Colon and Rectal Surgery Fellowship

Abstract

BACKGROUND:

As an increasing number of general surgery residents apply for fellowship positions, it is important to identify factors associated with successful matriculation. For applicants to colon and rectal surgery, there are currently no objective data available to distinguish which applicant attributes lead to successful matriculation.

OBJECTIVE:

The purpose of this study was to identify objective factors that differentiate colon and rectal surgery fellowship applicants who successfully matriculate with those who apply but do not matriculate.

DESIGN:

This was a retrospective analysis of colon and rectal surgery applicant characteristics.

SETTINGS:

Deidentified applicant data provided by the Association of American Medical Colleges from 2015 to 2017 were included.

MAIN OUTCOME MEASURES:

Applicant demographics, medical school and residency factors, number of program applications, number of publications, and journal impact factors were analyzed to determine associations with successful matriculation.

RESULTS:

Most applicants (n = 371) and subsequent matriculants (n = 248) were white (61%, 62%), male (65%, 63%), US citizens (80%, 88%) who graduated from US allopathic medical schools (66%, 75%). Statistically significant associations included graduation from US allopathic medical schools (p < 0.0001), US citizenship (p < 0.0001), and number of program applications (p = 0.0004). Other factors analyzed included American Osteopathic Association membership (p = 0.57), university-based residency (p = 0.51), and residency association with a colon and rectal surgery training program (p = 0.89). Number of publications and journal impact factors were not statistically different between cohorts (p = 0.067, p = 0.150).

LIMITATIONS:

American Board of Surgery In-Training Examination scores, rank list, and subjective characteristics, such as strength of interview and letters of recommendation, were not available using our data source.

CONCLUSIONS:

Successful matriculation to a colon and rectal surgery fellowship program was found to be associated with US citizenship, graduation from a US allopathic medical school, and greater number of program applications. The remaining objective metrics analyzed were not associated with successful matriculation. Subjective and objective factors that were unable to be measured by this study are likely to play a determining role. See Video Abstract at http://links.lww.com/DCR/B415.

Seeking advanced training in a surgical specialty on completion of general surgery residency has become increasingly common.¹ Of the possibilities, colorectal surgery has remained a popular choice. Applicants may be drawn to the specialty because of the diversity and varying complexity of procedures, ranging from management of anorectal disease to complex oncologic resections and advanced minimally invasive surgery, as well as lifestyle considerations and the influence of mentors.^2,3

At this time, there are 63 colon and rectal surgery (CRS) training programs across the United States and Canada that offer advanced education and are recognized by the American Society of Colon and Rectal Surgeons.⁴ Fellowships in CRS are highly desirable, and 89% of program directors agree that the match has become more competitive in recent years.⁵ Based on National Residency Matching Program (NRMP) data averaged over the past 5 years, <1% of programs went unfilled and ≈75% of positions were granted to graduates of US medical schools.⁶ For matriculation in 2019, 61 programs offered 103 positions to 138 applicants entering the match. All of the positions were filled, with approximately a quarter of applicants failing to obtain a spot in a CRS fellowship.

With an estimated 75% to 80% of general surgery graduates pursuing fellowship training, there has been considerable interest in determining applicant characteristics that are associated with a successful match.^7,8 Although multiple studies have attempted to define these factors for various surgical specialties, including pediatric, endocrine, hepatobiliary, and surgical oncology, the vast majority of publications have been survey-based.^9–14 In CRS, survey studies have shown subjective characteristics such as performance at interviews and letters of recommendation to be important; however, there are no published reports on objective characteristics of applicants applying to a fellowship in CRS. We therefore aimed to identify measurable factors associated with successful matriculation to CRS fellowship using applicant data acquired through the Electronic Residency Application Service (ERAS). We hypothesized that successful matriculation is associated with graduation from US medical schools, American Osteopathic Association (AOA) membership status, training at a residency program associated with a CRS fellowship program, number of publications, and publications in journals with higher impact factors (IFs). These factors have been demonstrated to be associated with successful matriculation of surgical applicants to other surgical subspecialties and therefore we posit that this will also be true of applicants to CRS programs.^14,15

MATERIALS AND METHODS

Data Acquisition

Fellowship programs in CRS approved by the Accreditation Council for Graduate Medical Education participate in ERAS, a service provided by the Association of American Medical Colleges (AAMC). Data from the AAMC were obtained via ERAS for applicants to CRS fellowship programs for application years 2015–2017 through an existing contract between the AAMC and the National Institutes of Health. These data correspond with matriculation in 2016–2018. To protect the identities of applicants, data were collected and deidentified by members of the AAMC before examination and analysis by our group.

The AAMC confirmed whether an individual applicant matriculated into a CRS fellowship by cross-referencing the applicant ID number between ERAS and the fellowship program rosters using the Graduate Medical Education (GME) Track Residency Survey. For applicants that applied more than once during the specified time, only data from the most recent application were used in the analysis.

Information regarding residency characteristics and their academic affiliations were obtained using the American Medical Association Fellowship and Residency Electronic Interactive Database and from the official websites of each institution. Residency programs were classified as university-based, community-based, community-based/university-affiliated, or military. Programs were considered to have an affiliation with a CRS fellowship program if an Accreditation Council for Graduate Medical Education–accredited CRS fellowship is based at the same training site or if residents participate in a colorectal surgery rotation at a hospital with a CRS fellowship in postgraduate year 3 to 5. This information was obtained from the listed curriculum provided by the official websites of each residency program. For applicants who were part of a residency program that did not participate in the GME Track Residency Survey, academic institution and CRS fellowship affiliations are categorized as unknown.

Publication data were self-reported by applicants through the ERAS application. Applicants are required to sign a statement on submission, affirming that the information provided is accurate; however, no independent verification by the AAMC is performed. The number of articles was calculated for each applicant and the names of the journals were documented and linked to the applicant identifier. Journal IFs were obtained from Thomson Reuters Journal Citation Reports year 2017, the journal’s website, or InCites Journal Citation Reports through the National Institutes of Health Library. Newly established journals that did not have an IF calculated by Clarivate Analytics were not assigned an IF for our data analysis.

Applicants were classified into tiers based on the highest IF achieved by any of their publications, as described previously by Wach et al.¹⁴ Tier 1 was defined as publication in a journal with an IF that was unknown, not yet established, or <2.5. Tier 2 represents applicants with a publication in a journal with a documented IF of 2.5 to 9.9. Tier 3 includes any applicant with an article published in a journal with an IF ≥10. Applicants who did not report any journal publications were classified as tier 0.

Statistical Analysis

Statistical analyses were performed using SAS 9.4 (SAS Institute, Cary, NC). Applicants were stratified based on matriculation status to CRS fellowship programs. Descriptive statistics are displayed as a raw count and percentage. Potentially important characteristics were analyzed using Fisher exact test, χ² test, and ORs to identify any association with successful matriculation into a CRS fellowship. Data were excluded if unknown. Number of applications and publications per applicant was compared using the Wilcoxon rank-sum test. To determine whether there was an association between publication tier and matriculation, a Cochran-Armitage test for trend was applied.

RESULTS

Applicant Pool Characteristics

During application years 2015–2017, there ranged 53 to 56 accredited CRS fellowship programs participating in the match, with total available positions ranging between 93 and 95 per year.⁶ A total of 371 applicants were identified by the AAMC and then stratified into applicants who matriculated into a CRS fellowship and applicants who applied but did not matriculate (248 vs 123). Eleven applicants applied more than once during this period, of whom 6 eventually matriculated to a CRS fellowship. Of the total applicant pool, most applicants were white (61%), male (65%), US citizens (80%), and graduated from a US allopathic medical school (66%). The majority of applicants (54%) were trained in an academic university-based general surgery residency program, and 29% of applicants trained at a residency program associated with a CRS fellowship program. Detailed applicant demographics are depicted in Table 1.

Table 1.

Demographics of all applicants who applied to a CRS fellowship

Applicant attributes	All applicants, n (%)	Matriculants, n (%)	Nonmatriculants, n (%)
Sex
Women	129 (35)	92 (37)	37 (30)
Men	242 (65)	156 (63)	86 (70)
Citizenship
US citizen	296 (80)	218 (88)	78 (63)
Non-US citizen or nonpermanent resident	75 (20)	30 (12)	45 (37)
Race of US citizensa
Asian	55 (18)	42 (19)	13 (16)
Black	24 (8)	18 (8)	6 (7)
Hispanic	25 (8)	18 (8)	7 (9)
White	192 (63)	141 (63)	51 (63)
Otherb	10 (3)	6 (2)	4 (5)
Type of medical school
US allopathic	246 (66)	186 (75)	60 (49)
US osteopathic	18 (5)	8 (3)	10 (8)
International or Canadian	107 (29)	54 (22)	53 (43)
Member of AOAc	44 (18)	35 (19)	9 (15)
Other Nonmedical graduate degree
PhD	9 (2)	3 (1)	6 (5)
MBA	2 (1)	2 (1)	0 (0)
MPH	23 (6)	16 (6)	7 (6)
Otherd	55 (15)	39 (16)	16 (13)
Type of residency
University-based	200 (54)	146 (59)	54 (44)
Community-based/university-affiliated	24 (6)	18 (7)	6 (5)
Community-based	87 (23)	66 (27)	21 (17)
Military	9 (2)	8 (3)	1 (1)
Unknown	51 (14)	10 (4)	41 (33)
Residency associated with a CRS fellowship	108 (29)	80 (32)	28 (23)

AOA = American Osteopathic Association; CRS = colon and rectal surgery.

^aApplicants are able to select more than 1 race; therefore, the category totals do not equal the total number of applicants.

^bOther races include American Indian or Alaska Native and “Other Race” as selected by the applicant.

^cAOA membership includes US allopathic medical school graduates only; does not represent the entire total applicant pool.

^dOther nonmedical degrees include master’s level or higher degree of various studies.

Matriculant Characteristics

Of the applicants who matriculated into a CRS fellowship, the majority were white (62%), male (63%), US citizens (88%), and graduated from a US allopathic medical school (75%). Nineteen percent of the matriculants were AOA members compared with 15% of nonmatriculants. Twenty-four percent of matriculants had additional nonmedical degrees, including 3 with a PhD, 2 an MBA, 16 an MPH, and 39 with another master’s level degree of various studies. Fifty-nine percent of the matriculants were trained in a university-based general surgery residency program, and 32% were trained at a residency program associated with a CRS fellowship.

Factors Differentiating Matriculants and Nonmatriculants

Multiple potential factors were assessed to identify if any were associated with increased likelihood of matriculation into a CRS fellowship. Factors analyzed included sex, citizenship, race (US citizens only), type of medical school, AOA membership, type of residency, residency association with a CRS fellowship program, number of program applications, number of journal publications per applicant, and highest IF associated with a publication. Of these, one statistically significant factor that increased an applicant’s likelihood of matriculating into a CRS fellowship was attendance at a US allopathic medical school (OR = 3.15 (95% CI, 2.00–4.97); p < 0.0001). US citizenship was also found to be a statistically significant demographic factor (p < 0.0001), whereas race of US citizens and sex were not statistically significant (p = 0.87; p = 0.20). The number of programs to which each applicant applied was also found to be statistically different between matriculants and nonmatriculants, with a median number of 32 versus 25 programs (range, 3–56 vs 1–56; p = 0.0004; Fig. 1). AOA membership (OR = 1.31 (95% CI, 0.59–2.92); p = 0.57), university-based residency (OR = 0.82 (95% CI, 0.49–1.39); p = 0.51), and association with a CRS fellowship program (OR = 0.96 (95% CI, 0.57–1.64); p=0.89) were not found to be statistically associated with matriculation to a CRS fellowship (Table 2). Although the number of journal publications per applicant and IF tier were not different between the groups, the median number of journal publications per applicant who matriculated was 3 compared with 2 for those who did not matriculate (range, 0–32 vs 0–51; p = 0.067; Fig. 2A). The numbers of matriculants in publication tiers 0, 1, 2, and 3, were 44 (18%), 53 (21%), 127 (51%), and 24 (10%) compared with 31 (25%), 25 (20%), 56 (46%), and 11 (9%) in applicants who did not matriculate (p = 0.14; Fig. 2B).

FIGURE 1.

Number of program applications. The number of colon and rectal surgery program applications by applicants stratified by matriculation status. Boxes represent upper quartile, median, and lower quartiles. Whiskers represent ranges. Median number of programs applied to by nonmatriculants is 25 vs 32 for matriculants (p = 0.0004).

Table 2.

Applicant factors associated with successful matriculation to CRS fellowship

Factor	OR	95% CI	P
US allopathic medical schoola	3.15	2.00–4.97	<0.0001
AOA memberb	1.31	0.59–2.92	0.57
University-based residencyc	0.82	0.49–1.39	0.51
Residency associated with CRS fellowship	0.96	0.57–1.64	0.89

AOA = American Osteopathic Association; CRS = colon and rectal surgery.

^aUS allopathic medical schools compared with US osteopathic and international medical schools.

^bAOA membership of US allopathic medical students compared with US allopathic medical students without AOA membership.

^cUniversity-based residency programs compared with community-based, community-based/university-affiliated, and military programs.

FIGURE 2.

Number of publications and publication tiers. A, The number of publications by applicants to a colon and rectal surgery fellowship stratified by matriculation status. Boxes represent upper quartile, median, and lower quartiles. Whiskers represent ranges. Median number of publications for nonmatriculants is 2 vs 3 for matriculants (p = 0.067). B, Publication with highest impact factor (IF) journal stratified by matriculation status. Applicants are displayed as a percentage of their respective cohort. Applicants were classified into tiers based on publications in their highest IF journal. Tier 0 = no publications; tier 1 = unknown IF, IF not yet established, or IF<2.5; tier 2 = IF 2.5–9.9; tier 3 = IF ≥10. There was no significant difference between matriculants and nonmatriculants (p = 0.15) in this regard.

DISCUSSION

There has been a notable shift toward subspecialization in surgery over the last few decades.¹⁶ As specialization becomes commonplace, graduating surgical residents flood the applicant pool, making the application process increasingly competitive, expensive, and time-consuming. One recent study reported that CRS applicants typically attend 11 to 15 interviews and spend more than $5000.⁵ Given this, there is a need for data to guide applicants in judging the strength of their applications. Multiple studies report factors associated with successful matriculation to various fellowship subspecialties.^9–14 However, there are no objective data to guide potential CRS fellowship applicants on the competitiveness of their application, because the literature is limited to survey studies for this specialty. A survey of program directors conducted by the NRMP in 2016 evaluated factors believed to be important for selecting applicants for an interview and ultimately securing a high position during the ranking process.¹⁷ For CRS fellowship applicants, letters of recommendation, interviews, and American Board of Surgery In-Training Exam (ABSITE) scores were most consistently cited as important factors. Another study by Bailey et al⁵ again identified that program directors placed the most weight on the interview process, followed closely by letters of recommendation and telephone calls from colleagues on behalf of the applicant.

Although survey studies play an important role in evaluating subjective factors, the data can be easily influenced by responder bias, selection bias, recall bias, and small sample size. Our study fills the void in knowledge with objective data. Using the ERAS database, our data encompass all CRS fellowship applicants and are obtained directly from the fellowship application, protected from bias in recall. After evaluating multiple objective attributes using our data set, 2 statistically significant objective factors associated with matriculation to CRS fellowship were US citizenship and attendance at a US allopathic medical school. These factors are likely related, although multivariate analysis could not be performed given that the provided data were aggregated. Given the small number of applicants that attended a US osteopathic medical school, the driving force behind the negative association with matriculation and type of medical school is likely attributed to the applicants who attended an international medical school. Importantly, an additional factor that distinguished matriculants from nonmatriculants was the number of programs to which candidates applied. This can be used as a guide for residents to determine how many programs they should apply to for a successful match, while considering factors such as cost and time spent away from training.

Although our data found that attendance at a US allopathic medical school was associated with matriculation to a CRS fellowship, there was no difference in matriculation based on the applicant’s residency type or whether the program has an affiliation with a CRS fellowship. In the NRMP survey, limited value was placed on training in a university-based compared with a community-based residency program with only 13% of program directors considering it in the ranking process.¹⁷ However, the reputation of the residency program was found to be more important, with 63% citing it as a factor. Given these responses, it is not surprising that residency program setting was not found to be significant in our study. Although program reputation may be a better predictor of outcome, there is no objective standard to measure the merit of a program, and therefore it was not possible to evaluate that variable in this analysis. Affiliation with a CRS fellowship program may not be important, because sufficient exposure to specialists in CRS is common among most residency programs.

We identified the median number of publications in CRS applicants as 2 to 3, consistent with published data.⁵ Number of publications and IF tier were not associated with matriculation, demonstrated by the fact that applicants in the highest IF tier made up ≈10% of both matriculant and nonmatriculant cohorts, and some applicants with greater than 20 publications failed to match. Of note, whereas 88% of CRS fellowship program directors believe that publications are important in selecting candidates for an interview, only 13% cite it as an important factor in ranking.¹⁷ With these data together, it implies that, although the number of publications may be important for obtaining an interview, the applicant must still perform well at the interview to rank highly.

Although the objective attributes of candidates that we analyzed have been found to be important in other specialties, the lack of statistically significant associations with matriculation found in our data set strengthens the conclusion that subjective factors that are more critically examined during interviews, such as clinical acumen, personality, and recommendation letters, are likely the most influential factors in matriculation to a fellowship in CRS. This is the main limitation of our study. We are also limited in the depth of our analysis, because only aggregate data were provided to protect applicant identities. Although aggregate data can show general trends, a thorough understanding of the strength of each applicant and their outcome in the match cannot be measured. We are also unable to determine whether accepted applicants matched to their highest ranked programs, because these data are unavailable to the AAMC. In addition, program characteristics for 51 applicants are missing because the residency programs of these applicants did not participate in the GME Track Survey. The self-reported nature of journal publications and lack of verification are also recognized weaknesses. Although applicants submit this information to ERAS on their honor, multiple studies have proven that a substantial proportion of these publications cannot be verified.^18,19 Finally, ABSITE scores were not available through the AAMC. Given that CRS has been reported to place the highest emphasis on ABSITE performance when compared with other fellowship programs, this factor could have a significant impact on match rate that would not be illustrated by this study.²⁰

CONCLUSION

We found that objective data points provided by the AAMC as submitted in ERAS applications to CRS fellowships are not the major determinants of successful matriculation. Subjective data, such as interviews and letters of recommendations, as well as objective factors unable to be assessed by our data set, are likely to play a predominant role. Information from this study, coupled with the current literature, demonstrate that objective factors that are often used to judge the strength of a curriculum vitae, such as AOA membership and greater number of publications in high impact journals, are not the critical determinants of successful matriculation to CRS fellowship.