Fellowship training as a modifier of the surgical learning curve

Abstract

Purpose

To investigate the effects of fellowship training on the learning curve for cancer control after open radical prostatectomy.

Methods

The study cohort included 7765 prostate cancer patients who were treated with radical prostatectomy by one of 72 surgeons at four major U.S. academic medical centers between 1987 and 2003. Multivariable models were used to determine the learning curves for biochemical recurrence and surgical margins, separately for surgeons with and without fellowship training, with adjustment for standard prognostic variables.

Results

Initial results of fellowship and non-fellowship trained surgeons were similar (5-year probability of recurrence for first case 19.4% vs 18.3%, respectively; absolute difference −1.1%; 95% CI −5.5% to 3.0%; p=0.7). However, the rate of learning was faster among fellowship trained surgeons (p=0.006), resulting in superior cancer control overall for fellowship trained surgeons (p=0.001; difference 4.7%; 95% CI 2.6% to 7.4%). In contrast, fellowship trained surgeons started off with superior positive margin rates (p=0.005; 36% vs 42%; absolute difference 6%; 95% CI 1% to 10%), but there was no obvious difference in the subsequent learning curve (p=0.9).

Conclusions

The learning curve for biochemical recurrence depends on surgical training, whereas the learning curve for surgical margins does not. This suggests that improvements in margin rates result from reflection on specific aspects of surgical procedure, while improvements in biochemical recurrence occur by some general process of improved surgical technique. Further research into the mechanisms of surgical learning is warranted.

We have previously reported that a surgeon’s rates of cancer control after radical prostatectomy improve with increasing experience: the “surgical learning curve”. In our primary analysis, we found that a typical patient had a 17.9% risk of recurrence by five years if treated by an inexperienced surgeon with 10 prior cases, compared to a 10.7% risk if treated by a more experienced surgeon with 250 prior cases¹. We subsequently reported the learning curve separately by pathologic stage. In patients with organ-confined disease, the most common presentation in contemporary practice, there was an approximately 10-fold difference in recurrence rates between patients treated by the most and least experienced surgeons².

We are continuing to analyze our data to determine other modifiers of the learning curve. One obvious potential modifier concerns a surgeon’s education: it is plausible that how surgeons are trained leading up to their first case as attending surgeon will affect how they learn once their career begins. Differences in surgical training are likely subtle, and are poorly documented and thus difficult to study. The exception is fellowship training, a clearly documented and relatively unequivocal marker of a distinct difference in surgical training.

In this study, we aimed to address three hypotheses concerning the effects of fellowship training on the learning curve. First, we hypothesized that overall rates of biochemical recurrence differ between surgeons with and without fellowship training. We then hypothesized that this difference would result from two separate effects: fellowship surgeons would have initially superior results to those without fellowship training as a result of the fellowship experience; moreover, fellowship trained surgeons would have more rapid improvements in outcome with rising experience (a shorter learning curve) compared to their non-fellowship trained counterparts. We also sought to repeat our analyses using positive surgical margins as an endpoint, in the hope that any differences in learning curves would be informative as to the mechanisms of surgical learning.

Patients and Methods

Sources of data and study cohort

The study cohort has been previously described¹. In brief, the data set consisted of 7,765 patients with clinically localized prostate cancer treated by open radical retropubic prostatectomy (RP) between 1987 and 2003 at one of four institutions (Memorial Sloan-Kettering Cancer Center, Baylor College of Medicine, Wayne State University and the Cleveland Clinic). Patients receiving neoadjuvant or adjuvant therapy were excluded. All data were de-identified prior to analysis and the study received appropriate institutional review board approvals.

Outcomes

Patient follow-up was conducted according to accepted clinical practice at each institution. In general, this consisted of serum PSA measurements every 3–4 months during the first postoperative year, semiannually the second year, and annually thereafter. Cancer recurrence was defined as a corroborated rising PSA level > 0.4 ng/ml; in rare cases, secondary treatment was initiated for a detectable and rising PSA ≤ 0.4 ng/ml: this was counted as an event. Surgical margin status was defined as the presence or absence of tumor cells at the inked margin of resection in the prostatectomy specimen.

Explanatory variables

All surgeons were asked to describe any fellowship training and whether their current practice was academic. Fellowship training was defined in terms as having at least one year of studies after residency. Surgeons were defined as having an academic appointment if they were faculty members of the Urology Department at their respective institution.

Statistical methods

For each patient, surgeon experience was coded as the number of RPs conducted by the surgeon prior to the patient's operation. This number reflects total prior experience, including operations conducted at former institutions, and those for patients ineligible for analysis. Only a single billing surgeon was recorded for each operation: operations at which a surgeon assisted, such as during fellowship training, were not counted towards prior experience.

As length of follow-up is not independent of surgeon experience, we used a log-logistic survival distribution to model hazard over time in all analyses. We also adjusted for within-surgeon clustering using generalized estimating equations, by specifying the cluster option in Stata. We did not cluster by institution as all surgeons all practiced at a single institution during the timeframe of the study, and institution could not therefore affect a surgeon’s learning curve.

Our first hypothesis was that fellowship trained surgeons have overall lower recurrence rates than non-fellowship trained surgeons, that is, a patient has a lower risk of recurrence if treated by a surgeon with fellowship training. To test this hypothesis, we entered fellowship training (yes vs no) in a model, with adjustment for case mix and year of surgery. The variables used to adjust for case mix were preoperative PSA, pathologic stage (extracapsular extension, seminal vesicle invasion and lymph node invasion) and pathologic Gleason grade.

Our second hypothesis was that fellowship trained surgeons have better initial results than non-fellowship trained surgeons, that is, the risk of recurrence for the first patient treated by a fellowship trained surgeon is lower than if that patient had been the first patients treated by a surgeon without fellowship training. To test this hypothesis, we fit a model including surgeon experience and fellowship training, with adjustment for case mix and year of surgery. From the model, we estimated the 5-year probability of recurrence for a surgeon's first case. The confidence interval and p-value for the difference in 5-year probability of recurrence between fellowship and non-fellowship trained surgeons were estimated using bootstrap methods with 1000 replications.

Our third hypothesis was that fellowship trained surgeons learn at a faster rate than non-fellowship trained surgeons. To test this hypothesis, we performed an interaction analysis; an interaction between two variables exists if the effect of one variable depends on the level of the other variable. If the interaction term between fellowship training and surgeon experience is statistically significant, then the effect of surgeon experience on recurrence – the learning curve - is affected by whether or not a surgeon received fellowship training. The sign of the interaction term indicates whether learning was faster or slower for fellowship trained surgeons. The interaction analysis was performed with adjustment for case mix and year of surgery. We restricted this analysis to the 3415 patients treated by a surgeon with 216 or fewer previous surgeries, as the most experienced surgeon amongst those without fellowship training had a lifetime experience of 217 surgeries.

Surgeon experience was entered as a continuous variable in all models using restricted cubic splines to model its nonlinear relationship with outcome. Since the distribution of surgeon experience varied for fellowship and non-fellowship trained surgeons, we varied the position of knots for the cubic splines according to the analysis performed. For our second hypothesis, knots were placed at the quartiles. For our third hypothesis, where the range for surgical experience was limited, we placed a single knot at 100 prior surgeries.

To plot the learning curve for biochemical recurrence, we fit a model including surgeon experience, with adjustment for case mix and year of surgery, separately for fellowship and non-fellowship trained surgeons. We then predicted the 5-year recurrence-free probability at the mean level of covariates from the entire cohort of 7765 patients.

All analyses were repeated for the outcome of positive surgical margins, using logistic regression in place of survival models. Statistical analyses were conducted using Stata 9.2 (Stata Corp., College Station, TX).

Results

There were 72 surgeons in our series (Table 1). The majority of surgeons (53, 74%) were non-fellowship trained. There was a strong association between training and surgeon experience: the median number of total cases performed by fellowship trained surgeons was 296, compared to only 32 for non-fellowship trained surgeons. In our series of 7765 patients, 82% were treated by fellowship trained surgeons. Only two non-fellowship trained surgeons performed > 100 total cases (total cases = 120 and 217). In contrast, 6 fellowship trained surgeons performed > 500 total cases (total cases = 535, 614, 710, 874, 1337, and 1979).

Table 1. Characteristics of surgeons with and without fellowship training.

	Fellowship trained
	Yes	No
Number of surgeons (%)	19 (26%)	53 (74%)
Surgeon experience1
Mean	443	36
Median	296	32
Interquartile range	83, 613	9, 49
Maximum	1978	216
Number of patients treated2 (%)	6371 (82%)	1394 (18%)

^1.Total number of cases treated as primary surgeon, for each surgeon in the cohort. For instance, of the 53 non-fellowship trained surgeons, half (i.e. 26) had lifetime caseloads of 32 or more.

^2.Total number across all surgeons. For instance, 6371 patients in our cohort were treated by a fellowship trained surgeon.

Baseline characteristics of the cohort are given in table 2. Patients treated by non-fellowship surgeons were slightly older (p<0.001), had slightly higher Gleason grade (62% vs. 49% with Gleason 7 or higher, p<0.001), but slightly lower rates of locally advanced disease (34% vs. 31%, p<0.04).

Table 2. Characteristics of patients treated by surgeons with and without fellowship training.

Patients treated by a fellowship trained surgeon with more than 216 cases prior to the patient’s operation are excluded. Data are frequency (%) or median (interquartile range)

	Fellowship trained
	Yes	No
	N=2021	N=1394
Age at surgery (years)	62 (57, 66)	63 (58, 67)
Total PSA (ng/ml)	7.3 (5.0, 12.3)	6.9 (5.1, 11)
Clinical stage1
T1	784 (39%)	648 (46%)
T2a	590 (29%)	483 (35%)
T2b	231 (11%)	166 (12%)
T3/T4	388 (19%)	89 (6%)
Biopsy Gleason score
<=6	1442 (71%)	928 (67%)
7	470 (23%)	378 (27%)
>=8	109 (5%)	88 (6%)
Pathology Gleason score
<=5	173 (9%)	84 (6%)
6	848 (42%)	445 (32%)
7	866 (43%)	766 (55%)
8	88 (4%)	83 (6%)
>=9	46 (2%)	16 (1%)
Extracapsular extension2	632 (31%)	392 (28%)
Seminal vesicle invasion2	205 (10%)	165 (12%)
Lymph node metastasis2	75 (4%)	55 (4%)
Surgeon experience
0–49	398 (20%)	1004 (72%)
50–99	426 (21%)	270 (19%)
100–216	1197 (59%)	120 (9%)
Positive surgical margins	662 (33%)	569 (41%)

^1.Preoperative staging

^2.Based on pathologic analysis of the radical prostatectomy specimen.

Among all 7765 patients, there were 1256 recurrences. Median follow-up for patients without recurrence was 3.9 years. Our first hypothesis concerned whether, overall, fellowship training affects recurrence. With adjustment for case mix and year of surgery, fellowship training was significantly associated with recurrence (p=0.001). The estimated 5-year probability of recurrence for a patient with typical cancer characteristics treated by a fellowship vs non-fellowship trained surgeon was, respectively, 11.3% vs 16.0% (difference = 4.7%; 95% C.I. 2.6%, 7.4%). There was no obvious difference in recurrence between fellowship and non-fellowship trained surgeons if surgeon experience was included in the model (p=0.9), suggesting that fellowship trained surgeons do better on average because they are more experienced.

The learning curve for biochemical recurrence, stratified by training of operating surgeon, is shown in Figure 1. The estimates given are for a case with tumor characteristics typical for the sample as a whole, that is, mean values of covariates were used in the model. The results for fellowship and non-fellowship trained surgeons are initially similar, but improve steadily only for the former. It appears that the results for non-fellowship trained surgeons initially get worse, however, the change is very small and well within the 95% confidence intervals and so the learning curve should be seen as essentially flat. A single non-fellowship trained surgeon performed >120 cases: rates of cancer control for this surgeon were comparable to those of the fellowship trained surgeons. The surgeon concerned trained many years ago, before the current system of fellowship training was well-established, and his status is therefore somewhat open to question.

Figure 1. The learning curve for cancer control after radical prostatectomy, separately for surgeons with and without fellowship training, with 95% confidence intervals.

Predicted probability of freedom of biochemical recurrence (BCR) at 5 years, calculated from a statistical model, is plotted against surgeon experience. Probabilities are for a patient with typical cancer severity in the entire cohort of 7765 (mean PSA, pathological stage and grade). Black lines: fellowship trained; Gray lines: non-fellowship trained. The dashed gray line is for the single non-fellowship trained surgeon with greater than 120 surgeries. The confidence intervals are given by thin black and gray lines.

To test formally whether initial results vary by fellowship training, we calculated estimates for the probability of recurrence for a surgeon's first case. Five-year probability of recurrence for a surgeon’s first case was 19.4% and 18.3% for fellowship and non-fellowship trained surgeons respectively (difference = −1.1%; 95% C.I. −5.5%, 3.0%; p=0.7).

The interaction analysis for our third hypothesis comparing rates of learning included a subgroup of 3415 patients. There were 738 recurrences in this subgroup with median follow-up for patients without recurrence was 4.3 years. In a model including both groups, the interaction term between fellowship training and surgeon experience was statistically significant (p=0.006) and in the direction of reduced recurrence rates, confirming that fellowship trained surgeons learn faster than those without fellowship training.

The outlying surgeon in our analysis – a surgeon without fellowship training who had approximately twice the lifetime experience of the next most experienced non-fellowship trained surgeon, and who had cancer control rates comparable to the fellowship trained group – had an academic appointment. This raises the possibility that it is academic practice, rather than fellowship training, that is causally associated with the learning curve. This is especially so because of the strong concordance between the two (table 3): all fellowship trained surgeons practiced in an academic setting, compared to only 11 of 53 (21%) non-fellowship trained surgeons. Moreover, we found that, when we repeated our analyses using the predictor of academic vs non-academic in place of fellow vs non-fellow – and without controlling for fellowship training - academic practice was associated with a lower risk of recurrence (p=0.005; absolute risk difference at 5 years = 4.6%; 95% C.I. 2.1%, 7.6%). However, in the interaction analysis the difference between the learning curves for academic and non-academic surgeons was not statistically significant (p=0.11).

In order to explore further the relationship between fellowship training and academic practice, we repeated the interaction analysis for fellowship training and surgeon experience restricting the sample to the 30 academic surgeons. The results did not importantly differ from our original analysis: the interaction term between fellowship training and surgeon experience was statistically significant (p=0.03) and protective of recurrence. This appears to confirm that the key explanatory variable is fellowship training and that practicing in an academic environment does not, by itself, confer the ability to improve with increasing experience.

Figure 2 shows the learning curve for surgical margins, separately for surgeons with and without fellowship training. In contrast to recurrence rates, fellowship-trained surgeons start off with superior margin rates (35.9% for fellowship vs 41.5% for non-fellowship trained; absolute difference 5.6%; 95% CI 1.1% to 9.5%; p=0.005) but there are no obvious differences in the subsequent learning curve (p=0.9 for interaction term).

Figure 2. The learning curve for positive surgical margins, separately for surgeons with and without fellowship training, with 95% confidence intervals.

Predicted probability of a positive surgical margin, calculated from a statistical model, is plotted against surgeon experience. Probabilities are for a patient with typical cancer severity in the entire cohort of 7765 (mean PSA, pathological stage and grade). Black lines: fellowship trained; Gray lines: non-fellowship trained. The dashed gray line is for the single non-fellowship trained surgeon with greater than 120 surgeries. The confidence intervals are given by thin black and gray lines.

Discussion

In our initial paper, we reported that there is a learning curve for biochemical recurrence after radical prostatectomy, with improving cancer control as surgeons accrue surgical experience. In this paper, we provide evidence that this learning curve is found only in surgeons who have fellowship training, with the recurrence learning curve for surgeons without fellowship training being essentially flat. We also found that few surgeons without fellowship training ever developed a significant lifetime caseload. Consequently, overall cancer control outcomes were superior for fellowship-trained surgeons.

We made three additional observations that we believe are particularly informative about surgical learning. First, we found that recurrence rates for the first patients treated did not differ by fellowship training. Second, rates of positive surgical margins improved similarly with experience for the fellowship trained and non-fellowship trained surgeons. Third, initial positive margin rates were superior for those with fellowship training. Our observations suggest that there are different mechanisms by which recurrence rates and positive margins improve over time.

This certainly seems plausible given the clear and obvious difference between margins and recurrence with respect to feedback: a surgeon is told of the patient’s margin status relatively soon after surgery, and may be able to relate a positive margin to a particular aspect of the procedure; biochemical recurrences, in contrast, occur many years later, and the surgeon is unlikely to remember any specifics about the patient’s surgery.

Accordingly, we hypothesize that improvements in margin rates result from reflection on specific aspects of surgical procedure, while improvements in biochemical recurrence occur by some general process of improved surgical technique. It remains an open question why only fellowship-trained surgeons experience the second type of learning. For a start, it is unclear whether the fellowship training experience itself has a causative role in surgical learning, or whether surgeons who choose to go through fellowship are also predisposed to think critically about their surgical technique. But even if we presume that fellowship training causes changes in the learning curve, we still could do little but speculate on the exact mechanisms. What we demonstrate here is that the research question of how surgeons learn to learn is not simply an academic question pertinent to educational policy, but has direct consequences for cancer control.

Our results are based on a group level analysis and therefore do not necessarily apply to individual surgeons. For example, some surgeons without fellowship training appear to have better margin and recurrence rates than selected fellowship-trained surgeons. Indeed, one of the surgeons in our analysis who did not have fellowship training did develop significant experience: the results of this one surgeon did improve with increasing experience and become comparable to those of the group average of the fellowship trained surgeons.

Several other authors have examined the effects of training on the learning curve. These studies have generally had no comparison group, involved only a small number of surgeons and patients, and have often used perioperative characteristics such as complications and blood loss rather than patient outcome as the endpoint. For example, Brown and Sajadi examined the results of 32 radical prostatectomies conducted by a single surgeon following fellowship and concluded that “six months of fellowship training doesn’t prevent the learning curve”³. The exact opposite conclusion - “a 1-year fellowship … can eliminate the learning curve for laparoscopic bypass” – was drawn by authors of a study on bariatric surgery. Although this study included more surgeons than Brown and Sajadi (five rather than 1) and also more patients (500 rather than 32), again there was no comparison group of surgeons without fellowship training⁴. Oliak and colleagues did include a comparison group of sorts, by reporting the results of two surgeons, one with and one without fellowship training. But we deem as questionable their conclusion that “fellowship training improves perioperative outcomes during a surgeon’s early experience”, on the grounds that a comparison of just two surgeons may provide information only about which of the two has superior outcomes⁵. A second study on radical prostatectomy similarly concluded that fellowship-training was of benefit by examining the results of two surgeons treating a total of 66 patients⁶.

In contrast, we include a large number of surgeons (72) and patients (7765), directly compare the results of surgeons with and without fellowship training, and use as an outcome the very reason why patients presented to the surgeon: cure of prostate cancer. We also avoid what we see as a trivializing assumption that the learning curve can be “prevented”, which suggests that a surgeon’s results would never improve between the first procedure as attending surgeon and the final procedure of a long career. In our view, surgical learning curves can be different shapes and sizes, but none are likely to be flat.

In sum, we found a learning curve for cancer control after radical prostatectomy only for fellowship trained surgeons. Accordingly, we conclude that either fellowship training confers the ability to improve surgical technique or alternatively, that surgeons who chose a fellowship are those with a greater a propensity to reflect on and improve their technique. Given the very large differences in outcome between patients treated at different points on the surgical learning curve, there is a clear warrant for further research into the mechanisms of surgical learning, especially where this occurs in the absence of feedback as to outcome.