Mortality and Severe Complications Among Newly Graduated Surgeons in the United States

Structured Abstract

Objective:

To evaluate severe complications and mortality over years of independent practice among general surgeons.

Summary Background Data:

Despite concerns that newly graduated general surgeons may be unprepared for independent practice, it is unclear whether patient outcomes differ between early and later career surgeons.

Methods:

We used Medicare claims for patients discharged between July 1, 2007 and December 31, 2019 to evaluate 30-day severe complications and mortality for 26 operations defined as core procedures by the American Board of Surgery. Generalized additive mixed models were used to assess the association between surgeon years in practice and 30-day outcomes while adjusting for differences in patient, hospital, and surgeon characteristics.

Results:

The cohort included 1,329,358 operations performed by 14,399 surgeons. In generalized mixed models, the relative risk (RR) of mortality was higher among surgeons in their first year of practice compared to surgeons in their 15^th year of practice (5.5% [95% CI 4.1%–7.3%] vs. 4.7% [95% CI 3.5%–6.3%], RR 1.17 [95% CI 1.11–1.22). Similarly, the relative risk of severe complications was higher among surgeons in their first year of practice compared to surgeons in their 15^th year of practice (7.5% [95% CI 6.6%–8.5%]) vs. 6.9% [95% CI 6.1%–7.9%], RR 1.08 [95% CI 1.03–1.14]). When stratified by individual operation, 21 operations had a significantly higher relative risk of mortality and all 26 operations had a significantly higher relative risk of severe complications in the first compared to the 15^th year of practice.

Conclusions:

Among general surgeons performing common operations, rates of mortality and severe complications were higher among newly graduated surgeons compared to later career surgeons.

Mini Abstract

In this observational study of 1,329,358 operations performed by 14,399 surgeons, general surgeons in their first year of practice had a higher relative risk (RR) of mortality (5.5% vs. 4.7%, RR 1.17 [95% CI 1.11–1.22]) and severe complications (7.5% vs. 6.9%, RR 1.08 [95% CI 1.03–1.14]) compared to surgeons in their 15^th year of practice.

Introduction

Graduation from residency is generally accepted to imply readiness for independent practice. However, there are growing concerns that surgical residency may not adequately prepare trainees for this transition.^1,2 Surgical faculty perceive that residents rarely achieve meaningful autonomy prior to graduation.^3–5 Surgical fellowship program directors believe that recently graduated surgeons cannot safely perform even basic procedures.⁶ Many graduates themselves feel unprepared for independent practice.^7,8 These gaps in the preparation of surgeons for independent practice could have negative consequences for surgical patients.^9,10

Despite these subjective concerns, to date it is unclear whether patient outcomes are actually worse among newly graduated surgeons compared to more experienced surgeons.¹¹ Prior work analyzing the relationship between career phase and outcomes is limited. Most studies have investigated a single operation in isolation, whereas practicing surgeons are expected to perform a wide variety of operations.^12–15 Studies that have included multiple operations either did not assess outcomes among early career surgeons or analyzed operations with low complication rates.^16,17 To date no study has compared the outcomes of new and experienced surgeons performing a representative sample of the entire breadth of procedures typically performed by general surgeons.

To address this gap, we evaluated mortality and complications among early and later career surgeons performing common operations. We examined outcomes for a national cohort of Medicare beneficiaries undergoing common operations defined as “core procedures” by the American Board of Surgery, indicating that surgeons should be able to perform them safely and independently. For each operation, we assessed how patient outcomes changed with surgeons’ years in practice. We hypothesized that surgeons at the beginning of independent practice would have worse outcomes than surgeons later in their career.

Methods

Data Sources and Study Cohorts

We used national fee-for-service Medicare claims for every Part B Medicare beneficiary undergoing a general surgery inpatient operation between July 1, 2007 and December 31, 2019. Patients were identified from the Medicare Provider Analysis and Review (MedPAR) file linked to the surgeon-identified Medicare Carrier files. We specifically identified patients undergoing any of 132 operations defined as “core procedures” by the American Board of Surgery. Core procedures are those that general surgeons are expected to be able to perform safely and independently upon graduation from residency.¹⁸ Procedures were identified using Current Procedural Terminology (CPT) codes which were mapped to a taxonomy defined by the Surgical Council on Resident Education.¹⁹ Claims for procedures in the bottom one percentile of procedure volume were excluded. We also excluded claims for patients who were younger than 65 years old, since these patients are entitled to Medicare because of a permanent disability or end-stage renal disease, which was beyond the scope of this study. We excluded patients older than 99 years old and patients who had more than one procedure performed during their index admission. Based on these criteria, 26 (of 132 potential) core procedures were selected for analysis (Supplemental Table 1, Supplemental Figure 1).

Surgeons were identified using the National Provider Identifier (NPI) associated with each procedure in Medicare Carrier files. NPI numbers were then linked with data from the American Board of Surgery (ABS), American Medical Association (AMA) Masterfile, and the National Plan and Provider Enumeration System (NPPES) to obtain surgeon characteristics including medical school, residency program, residency graduation year, fellowship training, and specialty.¹⁹ Surgeons were excluded if they did not complete general surgery residency at an allopathic residency program in the United States or if they completed additional fellowship training after completing general surgery residency. Surgeons with missing data were also excluded. Surgeons in practice for more than 35 years were excluded due to small sample size threatening anonymity. This resulted in a cohort of general surgeons in independent practice between 1 and 35 years.

Outcomes and Explanatory Variables

The primary outcomes assessed were mortality and severe complications within 30 days of surgery. Mortality was defined as patient death within 30 days of surgery ascertained from the Medicare Beneficiary Denominator file. Clinically significant severe complications were identified using International Classification of Diseases 9^th and 10^th Edition (ICD-9/10) diagnosis codes based on prior work examining postoperative complications in administrative claims.^20,21 These complications included pulmonary failure, pneumonia, myocardial infarction, deep venous thrombosis and pulmonary embolism, renal failure, surgical site infection, gastrointestinal bleeding, and hemorrhage. Complications present on admission were excluded from the definition of severe complications. Because this indicator identifying complications present on admission was only available in Medicare data from 2010–2019, complications were evaluated among this subset of patients while mortality was evaluated for the full cohort.

Patient characteristics included age, sex, race, socioeconomic status, and Elixhauser comorbidity index. Socioeconomic status was derived from the median household income of a patient’s zip code, divided into quintiles, with a binary variable for whether a patient’s zip code was in the top three quintiles of median household income. Procedural characteristics included the procedure itself and surgical priority (elective vs. urgent/emergent). Surgeon characteristics included years in practice and presence of an assistant surgeon. Surgeon volume was not included since our goal was to identify differences in outcomes between early and later career surgeons independent of operative volume, which is itself mediated by career phase. Hospital characteristics included hospital location (urban vs. rural), number of beds (<350 beds vs. ≥350 beds), intensive care unit (ICU) availability, nurse-to-bed ratio, and the ratio of Medicaid to inpatient days.

Statistical Analysis

Descriptive statistics were calculated for patient, surgeon, and hospital characteristics. Generalized additive mixed models were used to assess the association between years in practice and outcomes. Generalized additive mixed models are similar to generalized linear mixed models, but use penalized regression to allow for a non-linear (i.e., non-constant) relationship between surgeon years in practice and patient outcomes without overfitting.²² Prior work demonstrates that these models are consistent with traditional mixed modeling methods but more accurately assess associations that may change over time, which was key to examining the relationship between career phase and outcomes.²³ Model specifications included patient age, sex, race, socioeconomic status, comorbidity score, surgical priority, year of operation, operation, presence of assistant surgeon, hospital location, number of beds, ICU availability, nurse-to-bed ratio, and ratio of Medicaid to inpatient days as fixed effects; procedure category as a random effect; and surgeon years of experience as a smoothing term. Models were then used to estimate marginal means and predicted probabilities of mortality and complications at surgeon years 1 and 15, and a pairwise comparison was performed to calculate the relative risk of each outcome at year 1 vs. year 15. Analysis of outcomes was limited to surgeons’ 1^st to 15^th years of independent practice, since the goal of this study was to understand trends in outcomes during the early career, and prior work has shown that other factors such as cognitive decline and functional impairment affect later-career outcomes, which would potentially confound our analysis.¹⁶

All statistical analyses were performed using R version 4.0.3 (R Core Team, 2020).²⁴ Tests were two-sided and significance was set at α = 0.05. This analysis of de-identified administrative claims data was deemed exempt from regulations by the University of Michigan Institutional Review Board.

Results

The cohort included 1,329,358 operations performed by 14,399 surgeons (Table 1). Median (interquartile range [IQR]) patient age was 76 (71–82) years, 572,990 (43.1%) patients were male, and 1,158,802 (87.2%) patients were of white race. Median years of surgeon experience was 17 (IQR 10–24) years. Individual operation volume is displayed in Supplemental Table 2.

Table 1 –

Cohort demographics and characteristics

Characteristic	N (%) Total = 1,329,358
Patient Characteristics
Age (median (IQR))	76 (71–82)
Male Sex	572,990 (43%)
White Race	1,158,802 (87%)
Top Three Quintiles of SES	873,769 (66%)
Elixhauser Comorbidity Index (median (IQR))	6 (0–15)
Emergency Admission	839,777 (63%)
Surgeon Characteristics
Surgeon Years of Experience (median (IQR))	17 (10–24)
Had Assistant Surgeon	142,162 (13%)
Hospital Characteristics
Urban Hospital	1,287,817 (97%)
Hospital Beds > 350	507,598 (38%)
Hospital Has ICU	1,265,138 (95%)
Hospital Nurse/Bed Ratio (median (IQR))	1.81 (1.41–2.14)
Hospital Medicaid/Inpatient Days Ratio (median (IQR))	0.14 (0.08–0.21)

SES = socioeconomic status (derived from median household income per zip code); ICU = intensive care unit; IQR = interquartile range. Value represent N (%) unless otherwise specified.

Risk-adjusted mortality and complications decreased with surgeon years of experience (Figure 1). Across all included procedures, mortality decreased from 5.5% (95% CI 4.1%–7.3%) among surgeons in their first year of practice to 4.7% (95% CI 3.5%–6.3%) among surgeons in their 15^th year of practice. This equates to a relative risk of mortality of 1.17 (95% CI 1.11–1.22) in the first vs. the 15^th year of practice. Similarly, severe complications decreased from 7.5% (95% CI 6.6%–8.5%) among surgeons in their first year of practice to 6.9% (95% CI 6.1%–7.9%) among surgeons in their 15^th year of practice. This equates to a relative risk of severe complications of 1.08 (95% CI 1.03–1.14) in the first vs. the 15^th year of practice. Unadjusted mortality and complications are displayed in Supplemental Figure 2.

Figure 1: Risk-adjusted 30-day mortality and severe complications.

Adjusted rates estimated using a generalized additive model including patient age, sex, race, socioeconomic status, comorbidity score, surgical priority, year of operation, procedure, presence of assistant surgeon, hospital location, number of beds, ICU availability, nurse-to-bed ratio, and ratio of Medicaid to inpatient days as fixed effects; procedure category as a random effect; and surgeon years of experience as a smoothing term.

For each of the 26 included procedures, the relative risk and predicted probabilities of mortality and complications between the first and 15^th years of practice are displayed in Figure 2. Compared to the 15^th year of practice, in the first year of practice the relative risk of mortality was significantly higher for 21 individual procedures (representing 96.1% of operation volume) and the relative risk of severe complications was significantly higher for all 26 individual procedures (representing 100% of operation volume).

Figure 2: Relative risk and predicted probabilities of 30-day mortality and severe complications in year 15 compared to year 1. Procedures in descending order of observed incidence of 30-day mortality.

Relative risk estimated using generalized additive models using patient age, sex, race, socioeconomic status, comorbidity score, surgical priority, year of operation, procedure, presence of assistant surgeon, hospital location, number of beds, ICU availability, nurse-to-bed ratio, and ratio of Medicaid to inpatient days as fixed effects; procedure category as a random effect; and surgeon years of experience as a smoothing term.

Discussion

In this national study of general surgeons, mortality and severe complications were higher among newly graduated surgeons compared to later career surgeons. Specifically, the relative risk of mortality and complications was significantly higher in the first year of practice compared to the 15^th year of practice, even after adjusting for differences in patient, hospital, and surgeon characteristics. This relationship held true for individual procedures, with 21 procedures having a higher relative risk of mortality and all 26 procedures having a higher relative risk of severe complications among newly graduated surgeons. Importantly, all procedures analyzed in this study are defined as “core procedures” by the American Board of Surgery, meaning that trainees are expected to be able to perform these operations safely and independently upon graduation from residency.

To our knowledge, this is the largest study to date to substantiate growing concerns about surgical resident preparedness upon graduation. Over the last decade, a number of studies have suggested that graduating surgical residents may be unprepared to operate independently.³ Although these concerns have compelling face validity given observed reductions in trainee autonomy, decreased patient care responsibilities during training, and reductions in operative exposure during residency, it has remained largely unknown whether the outcomes of newly graduated surgeons differed from those of later career surgeons.^25,26 The current results suggest that these subjective concerns are mirrored in objective patient outcomes among early career surgeons. Furthermore, analysis of individual procedures revealed that it was not solely procedures with the highest complications and mortality that improved with more years in practice. Rather, procedures across the spectrum of mortality and complications – such as gastrectomy, appendectomy, and mastectomy – had a significantly higher relative risk of complications earlier in the career phase.

This study, which is notable for its size and breadth, expands upon prior work investigating the effect of career phase on surgeon outcomes. Smaller studies investigating individual surgical procedures in isolation have also demonstrated that surgeons in their first year of independent practice have higher complication rates.^12,27 Most similar to the current study, Kelz et al.¹⁷ analyzed Medicare claims for nearly 700,000 patients treated by over 8,000 surgeons performing 8 procedures to assess differences between new and experienced surgeons. Although the authors found that observed 30-day mortality was higher among new surgeons, this difference became nonsignificant after adjusting for differences in operation type, emergency status, and patient complexity. Conversely, we observed a significantly higher relative risk of mortality in the first versus 15^th year of practice, even after performing similar risk adjustment. These conflicting findings are likely related to our inclusion of a wider variety of procedures of varying complexity, whereas the majority of procedures analyzed by Kelz et al. were those for which residents are known to be well prepared prior to graduation.³ Additionally, although their study found no mortality difference, they still found that new surgeons had significantly higher adjusted rates of readmission or death, length of stay, intensive care unit usage, and 30-day resource costs. Importantly, our results provide strong support for these previous findings by demonstrating that the rate of severe complications is also higher among newly graduated surgeons.

These results invite the important question of whether these trends are modifiable. On the one hand, worse outcomes at the beginning of independent practice suggest that the training surgeons receive during residency could be enhanced to improve early career outcomes. On the other hand, some may argue that it is unrealistic to expect a newly graduated surgeon to achieve the same outcomes as a more seasoned surgeon. Indeed, surgeons are expected to become more proficient over the course of their career.^28,29 For a lifelong practice such as surgery, will patient outcomes inevitably be worse early in a surgeon’s career? Put another way, are we simply observing the “learning curve” of surgical practice? For example, it may be the case that newly graduates surgeons are not as skilled as more experienced surgeons, but that they are nevertheless “safe” surgeons. Even if this is the case, patients still expect that graduating surgeons are as prepared as possible for independent practice. The current results highlight a potential opportunity to improve outcomes precisely where they are most deficient, although future studies assessing the safety of new graduates should be undertaken.

A number of initiatives are already underway that may improve the outcomes of early career surgeons. For surgical training, the American Board of Surgery has begun to implement a more competency-based training model that would rely on “entrustable professional activities” (EPAs) to assess resident preparation for independent practice.³⁰ EPAs specifically evaluate whether a trainee can competently and independently perform a given clinical task, such as evaluation and management of a surgical pathology.^31,32 By focusing on competency and independence rather than case volume alone, this new training model may smooth the transition to independent practice and directly address the gaps observed in the current study. These changes will likely need to be coupled with gradually increasing resident autonomy as well, which has decreased in recent decades and may underlie unpreparedness at graduation.³³ Lastly, the results of this study suggest that increased supervision of newly graduated surgeons by more experienced surgeons may be beneficial. Indeed, the relatively new field of “surgical coaching” recognizes that training does not end upon graduation from residency, and that better outcomes can be achieved when surgeons are given in-depth feedback and assessment by senior colleagues.^34,35 This may serve as a practical model to further mitigate adverse outcomes in the early career.

This study has several limitations. Although inferred, this study did not explicitly assess the relationship between early career outcomes and the quality of surgical training. For example, it is possible that training adequately prepares trainees to be safe surgeons, but that other factors underlie increased complications and mortality rates immediately following graduation, such as learning curves and lifelong improvement. Going forward, it will be critical to evaluate the relationship between objective assessments of in-training performance – which are becoming increasingly available – and post-graduation outcomes.^36,37 Another limitation is that the observational nature of this study introduces the possibility of confounding from selection bias. It may be the case that later career surgeons have more latitude in their case selection and that our models did not sufficiently account for how newly graduated surgeons may have to accept more complex and emergency cases due to their junior status. Indeed, Kelz et al.¹⁷ put forth this argument to explain the lack of a mortality difference in their analysis. Nevertheless, we used appropriate statistical models to account for a number of obvious confounders, including emergency status and patient comorbidities, and still observed worse adjusted outcomes among early career surgeons. Along these lines, given that the goal of this study was to provide a broad, generalized investigation into changes in surgeon outcomes over time, we did not assess the effect of more nuanced details, such as disease severity, patient frailty, the effect of patient race, and geographic migration of surgeons over time. It is well-recognized that Medicare claims also lack the granularity to comprehensively assess these details. Therefore, these analyses were beyond the scope of this study and are currently being pursued in work focused on these topics. Similarly, Medicare claims are also limited by some amount of incompleteness and missing variables, which is frequently seen in other studies using Medicare claims, as well as their limited ability to capture other factors associated with postoperative outcomes.³⁸ A third limitation is that although this study included 26 core procedures as defined by the American Board of Surgery, these results may not be entirely generalizable. There are currently 132 core and 85 advanced procedures defined by the American Board of Surgery, and the relationship between career phase and patient outcomes may differ among the procedures that we chose not include in our investigation. However, this study included over two dozen of the most common core procedures, which were also the most common procedures in Medicare claims. Additionally, the very large sample size of this study increases its power to detect very small changes in outcomes over time that are of unclear clinical significance. Additional work is needed to better characterize the clinical significance of these small but statistically significant trends. Conversely, this study may be underpowered to detect differences in procedures with smaller sample sizes, although we note individual differences in risk of complications across procedures of varying sample size. Because our study relied upon ABS data to identify surgeon characteristics, we were also limited to including only surgeons who completed allopathic rather than osteopathic training and were unable to include additional surgeon demographics such as age, sex, and race/ethnicity. Lastly, while the results of this study naturally imply that continued efforts to improve the quality of surgical training are merited, these results offer no information about which improvements may be most effective or how to implement them. Ongoing collaboration between the certifying and administrative bodies that oversee surgical training and investigators evaluating surgeon outcomes will be integral to ensuring that surgical training in the United States is of the highest possible caliber.

Conclusion

Among general surgeons performing common operations, mortality and severe complications were higher among newly graduated surgeons compared to later career surgeons. Specifically, compared to the 15^th year of practice, in the first year of practice the relative risk of mortality was 1.17 (95% CI 1.11–1.22) and the relative risk of severe complications was 1.08 (95% CI 1.03–1.14). Efforts aimed at better preparing graduating residents for the transition to independent practice may improve outcomes for their future patients.