Radiation Oncology Residency Selection: A Post-Graduate Evaluation of Factor Importance and Survey of Variables Associated with Job Securement

Abstract

Background

Medical students often choose to pursue a career in Radiation Oncology with limited meaningful exposure to the field. We previously identified factors which were most influential to an applicant’s rank list order. Here we sought to assess if residency graduates had differing views regarding those factors. We also polled recent graduates’ attitudes of the current job market.

Material and Methods

An anonymous, internet-based survey was developed and distributed to graduates of radiation oncology residencies from 2003–06 and 2012–15 to assess the importance of factors with regard to residency selection, training, and job securement and attitudes toward the job market within the United States.

Results

Responses were received from 198 of 848 (23%) of those invited to participate. The respondents were divided into two cohorts for analysis, an “early” cohort (2001–09) and a “contemporary” cohort (2010–16) Respondents recalled “quality of clinical training,” “perceived happiness of residents,” and “sense of community among faculty and residents” as the three most important factors influencing the rank list; post-residency the most valued factors of the residency experience were “quality of clinical training,” “geographic location,” and “faculty mentorship.” Factors that were assigned the greatest differential value in hindsight to influence the rank list included “faculty mentorship,” “willingness of faculty to call employer,” and “quality of alumni base.” 64% of respondents reported the job market to be difficult or very difficult. This perception was more common among contemporary graduates (p<0.05). 60% of respondents reported “far too many” or “somewhat too many” residency positions for the actual job needs in the U.S.

Conclusion

After training, residency graduates place higher value on factors in residency that can directly improve job procurement. This finding is more common among more recent graduates, potentially a result of the perception of a tightening job market with too many radiation oncologists in training.

Introduction

The process by which medical students become familiar with and ultimately choose to pursue a career in radiation oncology is often based upon limited exposure to the field during their formal medical training. While efforts have been made to increase formalized exposure to radiation oncology largely through the development of a medical student curriculum provided to rotating medical students in radiation oncology at select institutions, generalized exposure remains suboptimal (1–5). Further attempts at increasing oncology specific educational objectives during medical school are being pursued but not yet widely implemented (6–8). As a result, individuals entering the field of Radiation Oncology often possess a limited appreciation of aspects specific to residency programs, which ultimately would translate into a meaningful training experience and be generalizable to a successful career as a radiation oncologist.

While training programs may emphasize a set number of unique opportunities within their curricula, these may have minimal impact on medical students’ selection process during the residency match. To this effect, previous efforts to clarify the perspectives of medical students when composing residency program rank lists demonstrated that geographical location, perceived happiness of residents, and clinical research opportunities were amongst the most important factors (9). While medical student perspectives in this setting are interesting and insightful, the stance of residency graduates could provide an “experienced” perspective regarding those factors most important when choosing a residency program both with regard to training competency and job procurement.

This observational cohort study expands upon a previous analysis assessing medical student perspectives regarding the residency selection process, with an analysis of factors noted to be important by individuals having completed radiation oncology residency training. Factors perceived as important in job placement and satisfaction are also reported.

Materials and Methods

An anonymous, internet-based survey was developed to assess the importance of factors with regard to residency selection, training and job securement as well as attitudes toward the Radiation Oncology job market within the United States. To ensure validity, a pilot analysis was performed prior to survey distribution. The survey was distributed to a small “test” cohort with feedback provided regarding the survey structure, questions, and output to ensure that recorded data would be sufficient to answer the proposed questions. The survey was distributed via email to graduates of accredited radiation oncology training programs from the years 2003–2016 with cohorts identified from archived Association of Residents in Radiation Oncology (ARRO) directories from 2003 and 2012. Within this time frame, two eras were targeted: those who graduated residency between 2003–2006, and 2012–2015. A five-dollar gourmet coffee card incentive was awarded to individuals who completed the survey with departmental grant funding from both institutions. The survey collected demographic information in addition to responses pertinent to the subject matter of the survey. Responses were multiple choice, 5-point Likert-type scale 1=Not at all, 2=Slightly, 3=Moderately, 4=Quite, and 5=Extremely or 1=Much less important, 2=Less important, 3=Neutral, 4=More important, and 5=Much more important, and free response. Likert-type scales are reported as mean, and median with interquartile range (IQR).

The survey remained open from May 9th, 2016, to May 27th, 2016. Three reminder emails were sent. Study data were collected and managed using REDCap electronic data capture tools hosted at xxxxx (10). REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources.

Data were analyzed using Stata v14.1 (StataCorp LP, College Station, TX). Descriptive statistics are used to report general respondent characteristics. An early and contemporary cohort was defined as 2003–2006 and 2012–2015, respectively. There were ten respondents who served in a pilot cohort to offer feedback on the survey tool whose graduation years fell outside of these two cohorts; eight were grouped with the early period (with graduation years 2001–2009) and two with the contemporary period (with graduation year 2016). Statistical comparisons of survey responses were made between these two cohorts. Student T-test was used to test for differences between continuous variables, Chi-square was used to test for differences between categorical variables, and Wilcoxon Ranksum and Kruskall-Wallis non-parametric tests were used to test for differences between two or greater than two ordinal variables. This study was approved by the Institutional Review Board at both institutions as exempt.

Results

Demographics

Complete responses were obtained from 198 of 848 practicing Radiation Oncologists (23.4% response rate, 6.1% margin of error at 95% confidence interval). Respondent characteristics are presented in Table 1. Year of residency graduation ranged from 2001–2016, with the most represented years being 2003–2006 and 2012–2015. Median residency size was 8 (range 2–30). All fifty states were represented, with the most commonly included being Illinois (19, 9.8%), California (19, 9.8%), Texas (11, 5.7%), Massachusetts (11, 5.7%), Ohio (11, 5.7%), Pennsylvania (10, 5.2%) and Florida (10, 5.2%). When questioned about overall satisfaction with residency training, 152 (77%) responded “quite” or “extremely” satisfied and only 10 (5%) responded “slightly” or “not at all” satisfied (Table 1). Overall residency training satisfaction did not change with time, Likert-type median 4[4–5] for the early and 4[4–5] for the contemporary cohorts (p=0.21).

Table 1.

Demographics and respondent characteristics

	All n=198	2000s n=76	2010s n=121	p (2000s vs 2010s)
Gender
Male	140 (70%)	60 (79%)	79 (65%)	0.04
Female	58 (30%)	16 (21%)	42 (35%)
Academic degrees
MD	196 (99%)
DO	2 (1%)
PhD	23 (12%)
MBA	6 (3%)
MPH	5 (3%)
JD	1 (1%)
Other	7
Year of training (n=197)*		76	121
Size of residency program
2–6	48 (24%)	21 (28%)	27 (22%)	0.70
7–8	63 (32%)	20 (26%)	42 (35%)
9–12	53 (27%)	24 (32%)	29 (24%)
13–30	34 (17%)	11 (14%)	23 (19%)
Satisfaction with residency	4[4–5]	4[4–5]	4[4–5]	0.21
Not at all satisfied	2 (1%)	1 (1%)	1 (1%)
Slightly satisfied	8 (4%)	3 (4%)	5 (4%)
Moderately satisfied	36 (18%)	12 (16%)	24 (20%)
Quite satisfied	89 (45%)	31 (41%)	57 (47%)
Extremely satisfied	63 (32%)	29 (38%)	34 (28%)

^*One respondent omitted year of training

Importance of factors when formulating residency rank list

Respondents were asked to think back to how they ranked factors when initially deciding on a training program. Here respondents were asked to score, on a 5-point Likert-type scale, a list of factors they used when deciding on their rank order list for residency (1=not at all important, 2=slightly important, 3=moderately important, 4=quite important, 5=extremely important). The five factors deemed to be most important as determined by mean and median Likert-type responses were: “quality of clinical training,” “perceived happiness of residents,” “sense of community among faculty and residents,” “faculty mentorship,” and “perceived program prestige” (Figure 1a and Supplementary Table 1).

Figure 1.

Change in importance of residency selection factors. Median value shown by diamond, with interquartile range shown with bars.

A comparison between graduation cohorts was performed. The contemporary cohort valued lifestyle and research-promoting aspects of training more heavily than the early cohort. Factors which were ranked as more important in the contemporary cohort included “perceived happiness of residents” (p<0.01), “sense of community among faculty and residents” (p=0.06), “workload of residents” (p<0.01), “opportunities to attend paid yearly conferences” (p=0.01), “protected research time” (p<0.01), “clinical research and clinical trial development opportunities” (p<0.01), “publication record of residents by time of graduation” (p=0.01), “willingness of faculty to reach out/call prospective employers” (p=0.03), and “proton therapy available” (p=0.01), whereas “technology available” was more important among the early cohort respondents (p=0.01).

Top three valued factors from residency training program

Respondents were provided a list and asked to choose only the top three factors valued most from their residency training experience. “Quality of clinical training” was the most valued factor with 69% of respondents assigning it within the top three followed by “geographic location” (27%), “faculty mentorship” (26%), “sense of community among faculty and residents” (25%) and “perceived prestige” (25%) (Supplementary Table 2). Those factors which were least likely to be ranked within the top three were “size of residency program” (2%), “overall appearance of clinic and resident facilities” (1%), “resident book fund” (1%) and “proton therapy available” (1%). Comparison of the top three valued factors between the early and contemporary cohorts revealed slight differences. The contemporary cohort was less likely to identify “perceived prestige” (p=0.08) and “technology available” (p=0.02) in the top three most important factors, and more likely to identify “perceived happiness of residents” (p=0.08) and “workload of residents” (p=0.05) compared to the early cohort (Supplementary Table 2).

Reevaluation of factor importance during residency selection: post-residency assessment

Respondents were asked to evaluate the same factors and determine whether or not they would assign differential value towards these factors when making a rank list, having completed residency training. Responses were a 5-point scale with 1=much less important, 2=less important, 3=neutral, 4=more important, and 5=much more important. “Faculty mentorship,” “willingness of faculty to call employer,” “quality of alumni base,” “job placement of graduates,” and “quality of clinical training” were the top factors that respondents valued relatively higher with the experience of residency and post-graduate work (Figure 1b). Comparison between cohorts revealed that the contemporary cohort assigned higher value to “willingness of faculty members to reach out/call prospective employers” (p=0.01), and “protected time for research” (p=0.08), and assigned lower value to “proton therapy availability” (p=0.05).

First job out of residency

Respondents were asked to provide information regarding their first job upon graduating residency. 58% of first jobs were classified as academic, and 42% were classified as private practice (Table 2). The contemporary cohort reported finding a job as either “Difficult” or “Very difficult” more than the early cohort (46% vs. 18%, p<0.01). Gender and advanced degrees were not associated with ease of finding a job. When questioned regarding the desirability of first job, 59% indicated that the first job was a desirable job in a desirable location. It was more common for respondents from the early cohort to report securement of a desirable job in a desirable location than the contemporary cohort (70% vs. 54%, p=0.05). Regarding perceptions of the current job market, 73% felt that the market was saturated in desirable geographic locations, while 28% perceived saturation in academic practice, and 22% in private practice (with 78% of respondents indicating saturation in any of these areas). 7% responded that none of these areas were saturated, and 17% indicated “I don’t know.” More recent graduates were more likely to perceive the job market to be saturated in academics, private practice, or favorable geographical locations (p=0.04). Overall the contemporary cohort reported the current job market being saturated in any area more frequently than the early cohort (85% vs. 68%, p=0.01). The majority of respondents felt that finding a job was either difficult (45%) or very difficult (19%) in today’s job market, with no statistical different between the two cohorts. The majority of respondents reported that “there are somewhat too many” (41%) or “far too many” residency positions for the actual need (19%). Less than 5% of respondents felt that “there are somewhat too few” or “far too few” residency positions for the actual need. There were no statistically significant differences between cohorts (Table 2).

Table 2.

First job and market saturation.

		All n=198	2000s n=76	2010s n=121	p
First job setting (n=193)
Academic		112 (58%)	44 (58%)	68 (58%)	0.80
Private practice		81 (42%)	30 (42%)	50 (42%)
Ease of finding a desirable job after completion of residency training		3[2–4]	4[3–4]	3[2–4]	<0.01
How satisfied were you with your first job out of residency?		4[3–5]	4[3–5]	4[3–5]	0.85
First job out of residency (desirable job, desirable location)
I found a desirable job, and I work in a desirable location		117 (59%)	53 (70%)	65 (54%)	0.05
I found a desirable job, but I work in a less desirable location		46 (23%)	11 (14%)	34 (28%)
I found a less desirable job, but I work in a desirable location		23 (12%)	10 (13%)	13 (11%)
I found a less desirable job, and work in a less desirable location		11 (6%)	2 (3%)	9 (7%)
I did not have success getting a job and was forced to take an alternate route		0	0	0
Other		0	0	0
Ease of finding a desirable job now?		2(2–3)	2(2–3) Mean = 2.4	2(2–3) Mean = 2.2	0.07
Very difficult		38 (19%)	12 (16%)	26 (21 %)
Difficult		90 (45%)	32 (42%)	57 (47%)
Neutral	52 (26%)		21 (28%)	31 (26%)
Easy	15 (8%)		10 (13%)	5 (4%)
Very easy	3 (2%)		1 (1%)	2 (2%)
Ease of finding desirable job now?
Neutral or difficult	179 (91%)		65 (86%)	114 (94%)	0.04
Easy or Very easy	18 (9%)		11 (14%)	7 (6%)
Is the radiation job market saturated in US?*
Yes, in academic practice	55 (28%)		15 (20%)	40 (33%)	0.04
Yes, in private practice	44 (22%)		12 (16%)	32 (26%)
Yes, in desirable geographic locations	144 (73%)		46 (61%)	97 (80%)
No	13 (7%)		8 (11 %)	5 (4%)
I don’t know	33 (17%)		16 (21%)	17 (14%)
Is the radiation job market saturated in the US?
Yes (any)	154 (78%)		52 (68%)	101 (85%)	0.01
No/I don’t know	44 (22%)		24 (32%)	20 (15%)
Number of residency positions (n=189)
There are far too many residency positions for the actual need	38 (19%)		15 (20%)	22 (18%)	0.83
There are somewhat too many residency positions for the actual need	81 (41%)		27 (36%)	54 (45%)
There are an appropriate number of residency positions for the actual need	64 (32%)		23 (30%)	41(34%)
There are somewhat too few residency positions for the actual need	6 (3%)		4 (5%)	2 (2%)
There are far too few residency positions for the actual need	0 (0%)		0 (0%)	0 (0%)

^*Percentages add up to more than 100% because respondents could choose multiple responses

Factors regarded as most important for securing a job post residency

Respondents were provided a list and asked to choose only the top three factors that are the most important for a resident to secure a good job. Here, “perceived prestige of training program” was reported as the most common factor for obtaining a good job (74%). The next most commonly reported factors were “willingness of faculty to call employer” (48%) and “quality of clinical training” (38%). When comparing between cohorts, the “willingness of faculty to call employer” was perceived as more important by the contemporary cohort (p<0.01) whereas “quality of clinical training” and “name recognition of faculty members” were more highly valued in the early cohort (Supplementary Table 3). Comparisons were also recorded based upon respondents in academics and private practice. “Geographic location” of training program appeared to be more important for respondents in private practice (p=0.04), while “publication record of residents at time of graduation” was perceived as more important for those in academics (p<0.01) (Supplementary Table 3).

Discussion

To our knowledge this is the first formal investigation of the perceptions of practicing radiation oncologists regarding the most valued factors of a residency training program. This study complements a previously published analysis surveying medical students and recently matched residents with regard to radiation oncology training program selection (9), and also presents perceptions of the current radiation oncology job market. The results of this survey provide insight that may be useful for prospective applicants to the field (“What should I look for when making a rank list to help secure my ideal job?”) and to residency programs (“What are applicants looking for and how might we attract the best candidates?”) alike. In addition, residency programs may utilize the results to help graduates obtain jobs.

When asked which factors respondents valued most, in retrospect, at the time of residency selection, “quality of clinical training,” “perceived happiness of residents,” “sense of community within the training program,” “faculty mentorship,” “perceived program prestige,” and “geographic location” were the top six factors, similar to our previous study (9). Having completed residency and gained work experience, the factors identified in the “top 3 most valued factors” from residency training remained these six factors. This consistency in valuation before and after training provides reassurance that applicants are judging programs fairly, and the most important factors that influence the rank order list remain important well beyond the match. However, when practicing radiation oncologists were asked to reassign weight to the factors influencing their rank order decisions in retrospect, there were some changes in perception. Factors that respondents would place differentially higher weighting on included “faculty mentorship,” “willingness of faculty to call prospective employer,” “quality of alumni base,” and “job placement of graduates.” All of these factors have direct impact on job procurement, which may not be at the forefront of a residency applicant’s mind.

There did appear to be differences regarding what the more contemporary cohort of graduates valued in residency, compared to the earlier cohort, with “earlier” and “contemporary” being emphasized only loosely as both cohorts are relatively “contemporary.” When applying to programs, the contemporary cohort was more likely to prioritize possible lifestyle factors (“perceived happiness,” “sense of community,” “workload”) and academic opportunities (“yearly conferences,” “protected research time,” “clinical research opportunities,” “publication record”). In the context of the perception of the increasingly difficult job market, it may not be surprising for recent graduates to prioritize factors that may improve their competitiveness for jobs (e.g. research time, willingness of faculty to make calls). Alternatively, these data might reflect an increasingly academically motivated group of trainees, although the contemporary cohort was not more likely to accept a first job in academics than the earlier cohort. The only factor less highly valued within the contemporary cohort when applying to programs was “technology available.” The early cohort trained in the period many would argue witnessed the most rapid change in technology from 2001–2009 with the implementation of intensity modulated radiotherapy (IMRT), image guided radiotherapy (IGRT), and emergence of proton therapy centers as well as other advanced delivery and dosimetric technologies and this finding likely reflects the awareness of these changes. On the other hand, the increasing adoption of new technologies by departments prior to 2009 may have normalized any differences in utilization of technology across programs, reducing the need for the contemporary cohort to prioritize this factor over others.

Our survey based study also sought to gain insight regarding perceptions of the current job and residency training environment within the United States. When asked about the job market at the time of graduation, the early cohort reported an easier time securing a job. These differences between the cohorts suggest a potential tightening of the job market, consistent with the perception of chief residents in the most recent ARRO survey (11). When asked about the current job market, there was no difference between the two cohorts with the majority of respondents reporting a perception that finding a desirable job is difficult or very difficult, indicating that even the early cohort, many of whom are in hiring positions now, perceives a tightening job market. 73% of respondents reported that the job market was currently saturated in the U.S. in desirable locations, and 60% indicated there being too many residents in training for the actual need. This perception bears consideration in light of the recent expansion of residency slots (12), with 56% more training slots added from 2004 to 2015, and the concern that prior projections on actual demand for radiation oncologists may have been over-estimated (13,14). A decrease in the ratio of applicants to residency positions which has already occurred, and an increasingly saturated market may have the undesired effects of a decreased caliber of applicant, or unfilled residency positions, similar to the recent experience of the field of radiology (15). However, the process of regulating residency slots is not straight forward in large part due to the need to respect antitrust laws (13), and more creative solutions must be considered to balance the supply of trainees to demand of the market.

As a survey study, our analysis has inherent limitations. We polled a sample of residency graduates with a finite number of questions and established variables for purposes of analysis. Also it is unclear what percentage of residency graduates have information listed within the registries, which could potentially lead to a selection bias among years. As the response rate overall was low (~25%), although typical for a survey response rate, broad conclusions cannot be generalized to the entire population based upon this data. It is important to highlight that there could be some bias in responses of individuals based upon how long they have been in practice and the time from when they formulated their initial rank lists. As these individuals’ perspectives have potentially changed, the relative importance of various factors may have been somewhat altered in comparison to the time of rank list formulation. The “contemporary respondents” could also suffer from bias related to their proximity to the process of job securement, which could cause them to perceive the market as tightening as the experience is more fresh in their minds. Although this facilitates data analysis, it also limits the breadth of responses that a free response survey might allow (e.g. one respondent commented that spouse preference should have been included). Also, the limited number of responses, and the potential bias of those who responded versus those who did not, may reduce the generalizability of these findings. Ultimately these findings represent opinion, which should also be taken in the appropriate context.

Nevertheless, these data provide insight regarding attitudes of how residency programs are valued both before and after training. The respondents in this survey suggest that the job market in desirable locations is saturated, and more recent graduates describe more difficulty in securing a desirable job. As the specialty of radiation oncology continues to attract highly qualified candidates, information such as this will be to be useful to appropriately inform candidates and allow for a potentially more satisfying residency and young independent career.