Learning Curve of Robotic Assisted Pyeloplasty for Pediatric Urology Fellows

Abstract

Purpose

Little is known about the learning curve of robotic surgery for surgeons-in-training. We hypothesized that pediatric urology fellows could attain proficiency in robotic pyeloplasty, defined as an operative time equivalent to that of an experienced robotic surgeon, within the two-year time frame of fellowship.

Material and Methods

From 2006 – 2010, we performed a prospective cohort study of four pediatric urology fellows and one pediatric urology attending performing pediatric robotic pyeloplasty. The operative times and surgical outcomes of 20 consecutive robotic pyeloplasties performed by four pediatric urology fellows (n = 80 cases) and a random sample of 20 cases performed by the attending surgeon were recorded. Multivariate linear regression was used to determine the change in operative time for each case the fellows performed and to estimate the number of cases necessary for fellows to achieve the median operative time of the attending pediatric urologist.

Results

The fellows’ operative times decreased at a constant rate of 3.7 minutes on average per case (95% CI 3.0 – 4.3 min/case). Fellows were projected to achieve the median attending operative time after 37 cases. No operative complications or failed pyeloplasties occurred.

Conclusions

Operative times for robotic pyeloplasty performed by fellows consistently decreased with cumulative surgical experience. These data can be used to help establish benchmarks of robotic pyeloplasty in pediatric urology assuming an appropriate exposure to robotics and an adequate case volume.

Introduction

The introduction of robotic surgery has dramatically increased the choice of operations available to address common urologic diseases and congenital anomalies. Now, robotic approaches for urological surgeries such as radical prostatectomy and pyeloplasty are performed routinely. The introduction of these novel procedures has significantly increased the technical skills that urology residents and fellows must acquire. Additionally, the time available for learning new operations is finite, given work hour restrictions and the case volume. These stand as challenges to fellowship programs’ goal of producing proficient surgeons.

There is a growing body of literature on the “learning curve” associated with robotic urologic surgery. Nearly all published studies, however, pertain to operations performed primarily in adult patients and the definition of learning curve differs between studies.^1–3 The only study that addressed robotic surgery in children focused on the learning curve associated with attending surgeons acquiring proficiency in robotic pyeloplasty.⁴ To our knowledge, no studies have addressed the learning curve of robotic operations for surgeons-in-training.

Pediatric urologists report the ability to perform robotic surgery as an essential skill that should be learned during training.⁵ Understanding the time required for fellows to learn novel operations such as robotic pyeloplasty is critical given that pediatric urology fellowship is a fixed two-year period after which structured mentorship is limited. Ascertaining the learning curve for robotic pyeloplasty will allow pediatric urology fellowship programs to determine the optimal structure of the fellowship in order to allow trainees to acquire the requisite skills prior to completing training.

We performed a prospective cohort study to determine the learning curve for perdiatric urology fellows performing pediatric robotic pyeloplasty. We define the learning curve as the improvement in robotic console time that occurs with mentored operative experience taking into account surgical complications and outcomes. We tested the hypothesis that pediatric urology fellows have the potential to attain proficiency in robotic pyeloplasty during the timeframe of a pediatric urology fellowship. In doing so, we aimed to estimate the number of cases necessary for fellows to attain proficiency.

Methods

Study Design

We performed a prospective cohort study between 2006–2010 at a single institution. The cohorts were patients 18 years and younger with UPJO for whom robotic pyeloplasty was performed entirely by an experienced attending surgeon (PC), or by one of four pediatric urology fellows under the attending surgeon’s direct supervision. The fellow cases were a consecutive series of operations in which the fellow performed >75% of the console time. The portion of the case performed by the fellow changed as experience increased. Typically, the progression was renal dissection then anterior anastomosis followed by posterior anastomosis. The 20 attending cases were a random sample of cases performed during the study period in which the attending performed 100% of the case. This was most often due to unavailability of the fellow to participate in the operation. The sample of 20 attending cases was selected from the eligible cases using a random number generator.

Children aged 1–18 years with ureteropelvic junction obstruction (UPJO) were eligible for inclusion. Exclusion criteria included those in which a concurrent operation was performed (e.g. pyelolithotomy) and those in which a ureteral stent had been placed prior to the scheduled pyeloplasty. Pre-stented patients were excluded because it is the surgeon’s experience that prior stenting increases the difficulty of the operation and, at the time of the study, it was standard practice to place a stent, should it be needed, in an antegrade fashion at the time of the pyeloplasty.

The primary outcome was operative time, which was defined as robotic console time (minutes); time required for anesthesia, trocar placement, robot docking and closure were excluded from the analysis in order to reduce inherent variability in the outcome measure and focus the analysis on the question at hand: actual surgery time. Console “switches” were recorded at the time of occurrence on a spreadsheet. Secondary outcomes included surgical success, defined as stability or improvement of hydronephrosis on post-operative ultrasound and operative complications. All patients were followed for 2 years following pyeloplasty. The primary explanatory variable was surgical expertise (pediatric urology fellow vs. attending). Additional patient characteristics and operative variables were recorded including sex, age, side of the obstruction, placement of an antegrade ureteral stent during the pyeloplasty, and etiology of the UPJO (crossing vessel vs intrinsic narrowing vs fibroepithelial polyp vs failed pyeloplasty). Heineke-Mikulicz repairs were performed for intrinsic obstructions and dismembered pyeloplasties were performed for those due to fibroepithial polyps and crossing vessels. This study was approved by the local Institutional Review Board.

Statistical Analysis

The operative time for fellow cases was defined as the median operative times of the four fellows during sequential cases. The median value was chosen to reduce the influence of outlying values. Because fellows did not always perform the entire case, the total operative times for the fellows were estimated using the following method to adjust for the proportion of attending involvement: Estimated total operative time = Actual operative time − [(% of case performed by attending x median operative time of the attending) + (% of case performed by attending x individual fellow’s operative time for that case number)]. Unless otherwise noted, fellows’ operative time reflects this adjusted time.

Linear regression was used to determine the relationship between the number of fellow cases and operative time. The number of cases at which point fellows would achieve the attending surgeon’s operative times was estimated by the point on the x-axis (case number) at which the regression line crossed the median operative time for the attending. An interaction term (cases squared) was included in an alternative model to test whether the fellows’ operative times were increasing at a non-linear rate. Sensitivity analyses using the uncorrected operative times as the dependent variable were performed. Post-regression analytics confirmed that assumptions of linear regression models were met. Comparisons of the cohort of patients operated by the attending and those operated by the fellows were performed using Fisher’s exact test for nominal variables and Kruskal-Wallis analysis of variance and the Mann-Whitney U-test for continuous variables. Kruskal-Wallis was also used to determine if the proportion of the fellows’ cases performed by the attending differed amongst the four fellows. Linear regression was used to determine if the proportion of the fellows’ cases performed by the attending changed across the study period. Statistical analysis was performed with Stata 11 (StataCorp, College Station, TX).

Results

There were no intra-operative complications or pyeloplasty failures in either cohort. Compared to patients operated by the fellows, a higher proportion of patients in the attending cohort underwent robotic pyeloplasty for previous failed pyeloplasty and had stentless robotic pyeloplasties (p = 0.007); however, the etiology of the UPJO did not affect operative times for either the attending (p=0.2) or the fellows (p=0.7). The use of a stent did not affect operative times for the attending; however, the operative times were longer for fellow cases (9/80) in which a stent was used (p=0.01). Other variables were similar between two cohorts of patients (Table). Amongst the four fellows, there was no difference in the proportions of the case that the attending performed (p=0.26) or change in the proportion of the case the attending performed across the study period (p=0.24). Follow-up was similar between the patients in the attending and fellow cohorts (p=0.97).

Table .

Characteristics of patients operated by attending and fellows

	Attending (n = 20)	Fellows (n = 80)	p
Sex (%)			0.56
Female	9 (45)	37 (46.3)
Male	11 (55)	45 (53.7)
Age, median* (IQR)	59.5 (30 – 104)	87.5 (30 – 142)	0.09
Side n (%)			1
Left	14 (70)	56 (70)
Right	6 (30)	24 (30)
Etiology of UPJO (%)
Intrinsic obstruction	3 (15)	30 (37.5)	0.07
Crossing vessel	8 (40)	40 (50)	0.46
Fibroepithelial polyp	6 (30)	10 (12.5)	0.08
Failed open pyeloplasty	3 (15)	0 (0)	0.007
Antegrade Stent (%)			<0.001
No	12 (60)	9 (11.25)
Yes	8 (40)	71 (88.75)
Follow-up, median* (IQR)	23.5 (23 – 24)	24 (22 – 25)	0.97

^*Age of patients is in months.

Operative times are shown in Figure 1. The median and mean operative times for the attending urologist was 58 minutes (IQR 40.5 – 65.5 minutes) and 54.45 min (SD 16.68 minutes), respectively. Using median operative times as the dependent variable, fellows’ operative times decreased by an average of 3.7min/case (95% CI 3.0 – 4.3 min/case). Inclusion of the case squared variable revealed no evidence that this rate of decrease was non-linear. Operative times for fellows were projected to reach those of the attending once 37 cases have been performed (Figure 2). In a sensitivity analysis in which unadjusted operative times were used, the fellows’ operative times decreased at a constant rate of 3.1 minutes/case (95% CI 2.6 – 3.6 min/case) and were projected to meet those of the attending at 39 cases. We observed that the adjusted operative times were higher than those uncorrected for attending involvement; however, this difference was minimal by the twentieth case. The residuals of the individual fellow operative times from the regression line decreased up to case number 13. From case 14 – case 20, the variance of fellow operative times increased (Figure 3).

Figure 1. Robotic pyeloplasty operative times for pediatric urology fellows and attending.

The median operative time for the attending pediatric urologist was 58 minutes. A trend of decreasing operative time for each case performed was observed for the fellows but not the attending.

Figure 2. Rate of improvement for pediatric urology fellows performing robotic pyeloplasty.

The rate of improvement for the fellows was constant over the time period studied. Pediatric urology fellows were projected to achieve operative times of an experienced robotic surgeon after performing 37 robotic pyeloplasties. The horizontal line crossing the y-axis at 58 minutes is the median operative time for the attending.

Figure 3. The variability in the operative times for the fellows fluctuated over the study period.

The variability in fellows’ operative times decreased initially but increased after the 13^th case (red vertical line)

Discussion

We report the learning curve of robotic pyeloplasty for children with UPJO. We found that fellows, on average, improve by 3.7 minutes per case while maintaining excellent surgical results. Using the confidence limits of the regression coefficient, which reflects the boundaries of the “true” rates of improvement, fellows increased their speed by 3 – 4.6 minutes per case. These estimates, which are similar to projections observed in the sensitivity analysis, support the robustness of our conclusion that fellows, assuming proper case volume and supervision, have the potential to become proficient in robotic pyeloplasty within a two-year fellowship program.

To our knowledge, no studies have examined the effect of the cumulative experience on the operative time for operations performed by surgeons-in-training. Within urology, most studies have dealt with robotic prostatectomy, where the number of cases necessary to overcome the learning curve varies widely from 20 to over 250 cases, which is likely due to the disparate outcomes measured.^{5, 6} Currently, there is no universally accepted definition of learning curve. Depending on the operation described, outcomes as diverse as surgical time and, for cancer surgery, positive margin rates have been used to define surgical learning curves. We believe that if surgical time is used to define the learning curve, it must be assessed along with the surgical outcomes. We believe that surgical success is paramount and that operative time is meaningless and potentially harmful if operative results are compromised. There were no failures and no complications in the consecutive series of cases in either the fellow or attending cases.

Our observations should not be over-interpreted, as the environment and experience of the fellows may not be generalizable to all situations. All fellows had prior experience with robotic surgery in adult patients and were mentored by a single attending surgeon. The 3.7-minute decrease in operative time for each case performed may not be applicable to fellows without previous robotic or laparoscopic surgical experience. Also, although we expect that fellows improve at a demonstrable rate with increasing experience, our results do not demonstrate and should not be interpreted that fellows can achieve the expertise of an experienced attending surgeon after 30–40 cases performed in fellowship. Furthermore, is likely that the attending surgeon performed more difficult operations as suggested by the younger age and more challenging types of repairs in the attending cohort, both of which did not reach, but approached, the traditional level of statistical significance. Additionally, we hypothesize, but have not tested, that the mentorship provided by a single surgeon decreases the variability in operative times that naturally exist for the same fellow and between different fellows. Thirdly, robotic pyeloplasty volume differs between centers. We believe this variability illustrates the importance of developing a universal curriculum for surgical skills required for commonly performed pediatric urology operations. Similar to extirpative urologic surgery, we believe that “dry” labs using surgical simulation can be developed to ensure that basic and advanced robotic skills are mastered prior to operating on a patient.⁷ Surgical simulation has the potential to improve patient safety, increase surgical training efficiency, and decrease operating room costs.⁸ Future studies should determine the impact of a surgical skills curriculum on learning curves for minimally invasive operations commonly performed pediatric urology.

We do not know if the decrease in operative times we observed persists beyond fellowship and into independent practice. It is probable that the lack of direct supervision would “reset” the learning curve of a recently graduated fellow performing pediatric robotic pyeloplasty to a new starting point; however, reports from other surgical series suggests that operative experience during fellowship improves performance after completing training. For open radical prostatectomy, Bianco et al. demonstrated that fellowship training increases the rate with which practicing urologists achieve desirable outcomes.⁹ Cook et al. have also observed that laparoscopic experience during training affects post-mentorship performance and practice patterns.¹⁰ It is plausible that similar trends would be observed in pediatric robotic pyeloplasty. Future studies that address the impact of fellowship training on post-training performance are important given the effects of learning curves on health care costs and clinical outcomes. Steinberg et al. summarized 8 previously published series on robotic prostatectomy learning curves and estimated that the overall operating room costs associated with becoming proficient in robotic prostatectomy are over $200,000.¹¹ It may be that learning new surgical skills during training is the most cost effective alternative as the impact of trainees on operative time and clinical outcomes appears to be minimal.¹²

There are limitations to this study. First, our projections are estimates, the accuracy of which is unknown. Indeed, the increased variance in the operative times of the four fellows after the thirteenth case may reflect separation of surgical times for the fellows. It is possible that this is the point at which “faster” fellows continue to decrease their operative times whereas others fellows may be approaching their own ultimate average time. Assuming that operative time variability correlates directly with the degree of case difficulty, an alternative and potentially coexisting reason for the increased variance in operative times is that fellows, as their experience grew, were performing more difficult operations in the latter half of the recorded cases than they were during the initial period. Second, we estimated the total operative times for fellows by adjusting for the proportion of the case performed by the attending. While we believe it is not ethical to have fellows perform the entire operation irrespective of the clinical circumstances, this restriction reduces the validity of our results. However, as reported earlier, using the 95% confidence intervals for the rate of change in operative times provides a range for the expected number of cases necessary for pediatric urology fellows to become proficient in robotic pyeloplasty. Third, it is probable that other unmeasured factors affect individual fellow’s learning curves. Such factors include overall operative experience, including open pyeloplasties performed, previous robotic training in residency, robotic cases in which the fellows performed <75% of the case, and the level of experience of the nursing team involved in robotic pyeloplasty. Finally, it is critical that our results are validated at other institutions before we can define with precision the learning curve associated with pediatric robotic pyeloplasty.

Conclusions

Operative times for robotic pyeloplasty performed by pediatric urology fellows decreased by an average of 3.7 minutes per case. Others should validate these findings and efforts should be made to define a curriculum for robotic surgery in pediatric urology.