Perceptions of readiness for independent practice among graduating orthopedic surgery residents in Ontario in the last 30 years

Abstract

Background:

There is increasing concern regarding the lack of physicians and underresourcing of the medical system in Canada. The training of orthopedic surgeons has emerged as an area of particular concern. The purpose of this study was to gain insight into the outcomes of graduates of orthopedic surgery residency programs in Ontario in the last 30 years.

Methods:

We invited graduates of orthopedic surgery residency programs in Ontario from 1992 to 2020 to participate in our survey regarding their practice patterns and career choices. Participants were asked whether they believed their residency had prepared them for independent practice and were asked about their practice patterns after graduation, including whether they completed fellowships.

Results:

A total of 618 graduates met the inclusion criteria. We had a response rate of 40.9% (n = 253). A total of 62.8% of participants reported feeling ready to enter independent practice, which was less than the 80% expected threshold. This proportion varied by program and, overall, those who had graduated more recently reported feeling less ready. Nearly all participants had completed at least 1 fellowship, with most trainees having completed 2 fellowships. Earlier graduates were less likely to complete 2 or more fellowships. Completing a fellowship did not help with comfort in practice nor with earlier employment. Most respondents reported that their current surgical skills were primarily influenced by fellowship training, regardless of comfort level in entering practice directly out of residency.

Conclusion:

A substantial proportion of orthopedic graduates reported not feeling comfortable entering practice directly out of residency, with only 62.8% of participants reporting feeling ready for independent practice after graduation. Furtermore, graduates are incurring a significant opportunity cost completing 1 or often 2 fellowships. These findings necessitate an appraisal of our goals in residency education.

The goal of any orthopedic residency training program is to produce graduates who are capable of functioning independently. Residents typically are skilled and safe operators, with the literature supporting that orthopedic surgery performed by residents does not affect patient outcomes, only surgical time.^1–3 The specialty of orthopedic surgery has the largest number of trainees pursuing fellowship after training.^4,5 This is a substantial increase from decades ago, when only a small number of graduates pursued further training.⁶ Recent reports have shown that patients waiting more than 6 months for arthroplasty have a significant decrease in quality of life.⁷ With an approximate cost of $800 000 to train a surgeon,⁸ the utility of further training ought to be examined.

It is crucial to understand trainees’ motivations behind pursuing fellowships. Do trainees not feel adequately trained? If so, is that a true assessment based on increased case complexity and advanced techniques, or merely a case of institutionalized imposter syndrome? Another hypothesis, suggested in the literature, is that market forces have led to more trainees completing fellowships.⁹ We sought to gain insight into these questions with this study by examining the outcomes of graduates of orthopedic surgery residency programs in Ontario in the last 30 years.

Whereas other studies have surveyed graduating residents on their motivations for fellowship,⁹ data showing a cross-sectional analysis of all graduating orthopedic surgeons from the province of Ontario have been lacking. Our group previously published work showing that the number of surgeons being trained and retained has not kept up with the growing population; as such, determining the motivations behind fellowship choices was a natural follow-up research endevour.¹⁰

We hypothesized that most orthopedic surgery graduates do not feel comfortable entering practice out of residency and that this trend is exacerbated among those who graduated more recently. Furthermore, we hypothesized that those who graduated more recently are completing more fellowships and that this is motivated more by necessity than interest.

Methods

Survey population

We collected the names and contact information of all physicians and surgeons who graduated in Ontario from 1992 to 2020 with a specialty in orthopedic surgery. We used the publicly available physician search from the College of Physicians and Surgeons of Ontario (CPSO). We searched for all active and inactive members of the CPSO with the designation of orthopedic surgery.¹¹ We collected each physician’s name, year of graduation, school of graduation, and registered practice address. For surgeons no longer practising in Ontario, or for those whose address was not available, we searched for their location using Google and confirmed this information using their local medical regulatory body. Where available, physicians’ email addresses were recorded. This information was typically available for physicians working in academic institutions. Postgraduate offices of Ontario orthopedic residency programs were contacted with the rationale for our study and asked if they were willing to share their alumni database with us. We excluded residents with foreign funding.

Where possible, the survey was distributed electronically with a reminder sent 1 week later, and a second reminder sent 1 month after distribution. Participants were sent a unique link; they had an unlimited amount of time to complete and could go back to change their answers before completion. Each question was mandatory. For those who did not complete or start the survey, if it was not complete within 1 week after the 1-month reminder, it was considered not complete.

When email was not possible or a reply was not received within a month, a paper copy of the survey was mailed with return postage to the surgeon’s office address. Due to complexities of arranging international return mail, physical mail was sent only to surgeons in the United States and Canada. Electronic responses were stored behind a 2-factor authentication on Qualtrics servers.

Survey design

The survey was designed by S.N. and edited by co-supervisors B.A.L. and E.H.S. The survey was sent to all members of the Division of Orthopedic Surgery at Western University for input. Members were given a rationale for the study and were instructed to not fill out the survey but to simply provide feedback on the types of questions. We then edited the survey based on feedback received for ethics approval before sending it out for data collection (Appendix 1, available at www.canjsurg.ca/lookup/doi/10.1503/cjs.005124/tab-related-content).

Study participants were asked whether they believed their residency had prepared them for independent practice (Appendix 1). Participants were given 2 options indicating “yes” (i.e., “Yes, I could have entered practice directly out of residency” and “Yes, but I feel like doing a fellowship helped make things easier”) and 3 options indicating “no” (“No, but with trial and error in practice I could have figured things out,” “No, I would have needed help from colleagues early on,” and “No, I definitely needed to do a fellowship and extra training”).

Participants were also asked about fellowships completed, practice patterns, employment, and demographic information (Appendix 1).

Partial responses were analyzed when possible. Respondents who reported 2 half fellowships were coded as doing 2 fellowships, and those who did half a fellowship were recorded as completing 1 fellowship, to focus on the number of fellowships in terms of discreet numbers and not time.

Statistical analysis

For the question about whether participants believed their residency had prepared them for independent practice, we grouped all the responses into a binary yes/no response. We set an arbitrary, but reasonable, threshold of 80% of graduates feeling ready for independent practice. Responses were then analyzed using the χ² test. We analyzed effect of school using a standardized residual of greater than or equal to 2, or less than or equal to −2.

To group the survey items asking about motivations for fellowship, we ran a factor analysis that resulted in a 3-factor solution. The 3 factors were categorized as extrinsic (i.e., “There was a specific opening that required that fellowship,” “There were more job openings in that specific fellowship than others,” or “That (those) specific fellowship(s) had better renumeration than others”), discomfort (“This was not an area I felt I was fully comfortable in being independent in”), and interest (“This was an interest of mine”).

To examine whether physician’s year of graduation predicted their motivations for fellowship, we ran a regression analysis predicting motivation from year of graduation, scale (i.e., intrinsic, extrinsic, or discomfort), and their interaction.

To assess which factors predicted physicians’ ability to get a job directly out of training, we performed regression analyses when the predictor was a continuous variable (e.g., year of graduation) and χ² tests when the predictor was categorical (e.g., current country of practice). For ease of interpretability, we present all the statistics together, but each of these predictors was tested using a separate test given the relatively small sample.

To examine current surgical skills based on fellowship or residency experience, we ran a 2 × 2 ([% of skills learned from residency, % of skills learned from fellowship; within subject] × [readiness for practice out of residency (yes/no)]) repeated-measures analysis of variance procedure.

All data were analyzed using Statistical Package for the Social Sciences (SPSS) version 26.¹²

Ethics approval

The Western University Health Science Research Ethics Board reviewed and approved the study.

Results

A total of 618 graduates met the inclusion criteria, and we recorded email addresses for 439 (71.0%). Of the 618 identified graduates, 471 (76.2%) were practising in Canada, 144 (23.3%) in the US, and 3 (0.5%) in other countries.

Year of graduation ranged from 1992 to 2020 (mean 2008, median 2010, and mode 2015). Distributions of the number of graduates per year are shown in Figure 1.

Fig. 1.

Number of orthopedic surgery graduates from Ontario per year.

We had an overall response rate of 40.9% (n = 253) and a completion rate of 39.2% (n = 242).

Job directly out of training

Approximately one-third (30.5%) of participants got a job directly out of training. Year of graduation (p = 0.1), school of graduation (p = 0.4), current country of practice (p = 0.5), number of fellowships (p = 0.3), gender (p = 0.6), and locuming during fellowship (p = 0.7) did not predict the likelihood of a job directly out of training.

Self-reported comfort entering practice independently

When participants were asked whether they believed their residency had prepared them for independent practice, the overall result was significant, indicating that participants did not evenly split their responses across all 5 response options (p < 0.001). More participants than expected by chance (43.0%) reported that they felt ready but a fellowship made things easier. Fewer participants than expected by chance reported that they would have needed help from colleagues early on (7.8%) or that they were not ready but could have figured it out with trial and error (13.1%). Responses to the other options did not differ from what would be expected by chance.

When we grouped responses into a binary yes/no response, a total of 62.8% of participants reported feeling ready for independent practice out of residency, which was significantly different from the expected 80% (p < 0.001).

The number of respondents reporting feeling ready for practice varied as a function of time, with surgeons who graduated more recently feeling less ready (β = −0.05, standard error = 0.02, p = 0.01).

There was an effect of school on readiness to practice (p = 0.006) (Table 1). We found that graduates of the University of Toronto were marginally more likely to feel unprepared for independent practice than predicted by chance (51% of graduates).

Table 1.

Analysis of school and readiness to practice out of residency

School of graduation		Ready for practice?		Total
		No	Yes
McMaster University	Count	9	16	25
	Standardized residual	−0.1	0.1
NOSM University	Count	2	0	2
	Standardized residual	1.5	−1.1
University of Ottawa	Count	26	43	69
	Standardized residual	0.1	−0.1
Queen’s University	Count	5	15	20
	Standardized residual	−0.9	0.7
University of Toronto	Count	37	35	72
	Standardized residual	2.0	−1.5
Western University	Count	15	50	65
	Standardized residual	−1.9	1.4
Total	Count	94	159	253

Time after formal training to feel comfortable operating independently

Most physicians reported feeling comfortable operating independently within 0–6 months after completing their training (52.7%); 16.9% felt comfortable between 6 months and a year, 17.3% after 1–2 years, and 13.2% after 2 years. We combine responses indicating 2–3 years, 3–4 years, and 4–5 years because of the relatively small number of physicians reporting each of these.

Neither number of fellowships (p = 0.09) nor school of graduation (p = 0.07) predicted time after formal training to feel comfortable operating independently.

Those who graduated more recently were more likely to endorse feeling comfortable after 6 months to 1 year (p < 0.001) or after 1–2 years (p < 0.001) compared with 0–6 months; that is, as graduation year increased, it took physicians longer to feel comfortable practising independently. Those who graduated more recently were less likely to endorse feeling comfortable more than 2 years after graduation (p < 0.001) compared with less than 6 months.

Number of fellowships

Almost all participants (98%) completed at least 1 fellowship. The mode was 2 fellowships (n = 115), and 4 participants (1.6%) reported not completing any fellowships. There was no difference in the number of surgeons completing 1 fellowship (n = 109) or 2 fellowships (p = 0.7). Physicians who reported completing more fellowships did not report feeling more ready to practice (binary readiness-to-practice measure; p = 0.2).

We found no effect of school (p = 0.1), country of practice (p = 0.7), or gender (p > 0.9) on number of fellowships completed. Respondents who had graduated earlier were significantly less likely to complete 2 (p < 0.001) or 3 or more fellowships (p < 0.001) compared with completing 1 fellowship.

The most common fellowship completed was arthroplasty (42/253 respondents; 16.6%), followed by sports (35/253 (13.8%). Full results are shown in (Table 2).

Table 2.

Distribution of reported fellowships

Fellowship	No. (%) of respondents n = 253
Arthroplasty	42 (16.6)
Sports	35 (13.8)
Trauma	31 (12.2)
Upper extremity	28 (11.1)
Spine	20 (7.9)
Pediatrics	19 (7.5)
Trauma and upper extremity	13 (5.1)
Trauma and arthroplasty	9 (3.6)
Foot and ankle	9 (3.6)
Other	47 (18.6)

Motivations for fellowships

In our analysis of motivations for fellowship, we found that discomfort was the least common motivator and was significantly less often endorsed than interest (p = 0.048). However, participants were relatively more motivated by discomfort (v. interest) the later they graduated (p = 0.01).

The school physicians graduated from did not affect their motivations for seeking fellowship (p = 0.7).

Locuming during fellowship

Of the participants who responded to the question (n = 245), 38.4% locumed during fellowship. There was a significant effect of time (p = 0.002) and country (p = 0.001), with participants who graduated more recently being more likely to locum and with graduates employed outside of Canada being less likely to locum. School of graduation did not predict likelihood of locuming during fellowship (p = 0.1).

Current surgical skills based on fellowship or residency experience

Those who graduated more recently attributed a higher weight of their current skills to fellowship (p = 0.01). There was no effect of year of graduation on percentage of current skills learned from residency (p = 0.3).

There was no effect of school on percentage of skills attributed to fellowship (p = 0.5), nor residency (p = 0.2).

We observed a significant effect of skills gained from residency versus fellowship, with skills learned from fellowship being weighted 1.33 times higher than those of residency. We did not observe a significant effect of readiness for practice out of residency (p = 0.6).

These effects were superseded by a significant interaction (p < 0.001), with those feeling ready to practice out of residency reporting more skills learned from residency, than those who did not feel ready to practice out of residency (p < 0.001). Participants who reported not feeling ready to practice out of residency attributed a greater percentage of skills learned from fellowship than those feeling ready to practice out of residency (p < 0.001).

Discussion

Our key finding is that a substantial proportion of orthopedic graduates reported not feeling comfortable entering practice directly out of residency. We set a reasonable threshold of 80% of graduates feeling comfortable to practise, and our results showed that only 62.8% felt comfortable. Furthermore, those who graduated more recently were more likely to report not feeling comfortable entering practice. This is a concerning finding and worthy of future discussions.

We hypothesized that job availability would have influenced fellowship choices; however, our results did not show this. We found that graduates were more motivated by interest in the field. Whether this is true interest or simply justification in hindsight for decisions made in the past is unknown.

Another important finding is that many (approximately 50%) of our graduates completed 2 fellowships. Completion of 2 fellowships has a considerable opportunity cost to the health care system and the surgeon. In the US, most (> 90%) graduates pursue fellowship after residency,⁴ with only a small number (4.5%) of graduates in a 2018 study having completed 2 fellowships.¹³ Our results show that the driving motivator for pursuit of fellowship is self-reported interest. However, it is unclear why Ontario graduates are interested in pursuing 2 fellowships. Our study was not designed to differentiate motivators for a second fellowship, but this is a potential area of future study.

Our results raise the question of why orthopedic graduates lack a feeling of readiness for independent practice. The literature on comfort in entering practice is limited. Studies in the US on general surgery show that US graduates feel comfortable entering general practice.¹⁴ This finding that some orthopedic surgeons in Ontario do not feel ready needs further assessment. Our finding of graduates reporting discomfort entering practice directly out of residency must be interpreted with caution. This study was not designed to assess the nuances of the type of discomfort, such as discomfort related to an impostor syndrome or discomfort due to lack of adequate training and potential patient harm.

With regards to residency training, the years required to complete an orthopedic residency program have not changed in many years and currently remains at 5 years. As noted in this study and others, fellowship training of 1–2 years is standard, leaving us to question the duration of training needed, whether orthopedic residency needs to be more than 5 years, or whether rotations in residency need to be more focused. We also showed that graduates mainly attribute their current skills to those learned in fellowship and that those who graduated more recently are more likely to do so. Could we start residents in a focused practice while in residency, or is orthopedics evolving into a lengthier program? Why are more recent graduates attributing more of their skills to fellowship? Why are newer graduates feeling less comfortable? Is it because the complexities of orthopedic training have increased, because the quality of training has decreased, or because the intrinsic standards of quality have increased? These are all interesting areas for future discussion.

Another important finding from our study was a constant 20% of graduates emigrating to the US. This is a substantial number of graduates and represents a considerable loss to Ontario in terms of investment in training these graduates. This is a finding that decision-makers (e.g., the Ontario Ministry of Health and postgraduate training committees) must reach consensus on as to whether this is an “acceptable” attrition. Further analyses on factors leading to graduate emigration is currently underway and is of substantial interest.

Limitations

A limitation of this study is the data set. When searching the CPSO, we used the filter of certification of independent practice in orthopedic surgery. As such, individuals who did not report to the CPSO that they met the requirements for independent practice would not be captured. These would be individuals who, after residency, left Ontario and never returned. The residency offices of Western University, the University of Ottawa, and the University of Toronto were able to share their alumni list with us; we consider our database for those schools to be comprehensive. McMaster University stated that they had recently started to build their list and shared what they had. Queen’s University and NOSM University did not share their lists. As such, there is a chance that we did not capture graduates from those 3 schools.

The biggest limitation of this study is the 40% response rate. Although we view this as a success in gaining a 40% response rate among practising surgeons, it is important to keep this limitation in mind when interpreting the results. To our knowledge, this work is the first of its kind, and as such we view it as a springboard for further work on this relevant and important topic in the field of orthopedic surgery.

Conclusion

We found that a substantial proportion of orthopedic graduates, especially new graduates, reported not feeling comfortable entering practice directly out of residency, and that the average graduate completed 2 years of fellowship training after completing their residency. These findings necessitate an appraisal of our goals in residency education. Residency is currently a 5-year process, and it is vital that our graduates are properly prepared to enter practice.