Evaluation of the Learning Curve in Ileal Pouch–Anal Anastomosis Surgery

Abstract

Summary Background Data:

We define the learning curve required to attain satisfactory training in ileal pouch–anal anastomosis (IPAA) and identify possible differences in the learning curve for stapled and hand-sewn IPAA surgery. Various studies have addressed the differences in failure rate between stapled and hand-sewn IPAA, but there is no literature that evaluates the differences in attaining satisfactory training in each of these techniques.

Methods:

Data were collected from 1965 patients undergoing IPAA surgery by 12 surgeons in a single center between 1983 and 2001. Using ileoanal pouch failure as the primary end point, a parametric survival model was used to adjust for case mix (patient comorbidity, preoperative diagnosis, manometric findings, and prior anal pathology). A risk-adjusted cumulative sum (CUSUM) model was used for monitoring outcomes in IPAA surgery.

Results:

The 5-year ileal pouch survival was 95.6% (median patient follow-up of 4.2 years; range 0–19 years). Fifty percent of trainee staff demonstrated a learning curve in IPAA surgery. Having adjusted for case mix, trainee staff undertaking stapled IPAA surgery showed an improvement in the pouch failure rate following an initial training period of 23 cases versus 40 cases for senior staff. The learning curve for hand-sewn IPAA surgery was quantified only for senior staff who attained adequate results following an initial period of 31 procedures.

Conclusions:

The CUSUM method was a useful tool for objectively measuring performance during the learning phase of IPAA surgery. With adequate training, supervision, and monitoring, the learning curve in IPAA surgery may be reduced even further.

The study evaluates the learning profile in ileal pouch–anal anastomosis surgery using risk-adjusted cumulative sum analysis for survival data. The learning curve for senior staff was 40 cases for stapled and 31 cases for hand-sewn anastomosis. For junior staff, the learning curve was only quantified for stapled anastomosis (23 cases).

Surgical performance changes over time, especially immediately following the introduction of a new procedure or as a surgeon first practices a new technique. These changes in surgical performance may represent a “learning curve” which is a function of (1) the technical developments or refinements in techniques after the introduction of a new procedure; (2) surgeon familiarity with new techniques; and (3) changes in infrastructure such as better-trained assistance and improved postoperative care. These changes often lead to an improvement in surgical performance and have been used for the evaluation of the learning curve in the clinical and nonclinical literature.^1,2

The cumulative summation technique (CUSUM) is a method for monitoring surgical performance and credentialing the practice of medicine and delivery of health care.^3–5 Since the introduction of the restorative proctocolectomy with the formation of ileal pouch–anal anastomosis (IPAA) in 1978,⁶ there have been no studies that evaluate the learning curve in IPAA surgery. A number of studies have addressed the differences in outcomes between stapled and hand-sewn IPAA,^7–9 but there is no literature that evaluates the differences in attaining satisfactory training in each of these techniques. Using a novel risk-adjusted CUSUM methodology for survival data, the study aims to define the learning curve or case experience that might be required for a surgeon to become proficient with stapled and hand-sewn IPAA surgery.

PATIENTS AND METHODS

Participants

Patients undergoing restorative proctocolectomy with IPAA between February 1983 and December 2001 were identified through the institutional review board–approved Cleveland Clinic Ileal Pouch Database at the Department of Colorectal Surgery. Risk factors considered in the analysis included patient demographic characteristics, duration and extent of disease, patient comorbidity, previous operations, preoperative diagnoses, anorectal manometric findings, details on the surgical procedures, and postoperative pathologic diagnoses along with the early (within 30 days after surgery) and late complications. The primary outcome was ileoanal pouch failure, defined as excision of the ileoanal pouch, formation of a permanent ileostomy, or pouch-related mortality any time during the follow-up period.

Calculation of Pouch Survival

The ileal pouch failure rate was regarded as a time-dependent variable, and unifactorial survival was undertaken to identify individual risk factors related to IPAA survival. The parametric survival model accounted for censored data (ie, patients lost on follow-up or whose ileal pouch was in situ and functioning at the time of the analysis [last follow-up visit]). Risk factors with a univariate P value of < 0.25, were considered in the multivariate model. To maximize the information extracted from the predictor and response variables, the technique of multiple imputation was used to substitute for incomplete data.¹⁰ A multivariate parametric survival model based on the Weibull distribution was developed for stratifying the risk of pouch failure in patients undergoing IPAA surgery. The model calculated the individual patient probability of pouch failure (F_t) at a particular point in time (t) based on the patients’ risk factors and disease severity (xb) as shown in Table 1. The calculation was:

TABLE 1. Frequency Table of Patient Characteristics, Type of Procedure Performed, Outcome, and Follow-up of the Study Population

where a >0, b = the regression coefficients for the patients’ risk factors derived from the survival model. A nonparametric bootstrap resampling technique with 10,000 iterations was used to calculate standard errors and to correct bias in the parameter estimation.

The Risk-Adjusted Cumulative Sum (RA-CUSUM) Chart

The cumulative sum technique was originally developed during World War II as a quality-control test for ammunition production lines. It belongs to a series of techniques collectively known as sequential analyses,¹¹ which allow an observer to identify whether a production process is “in control” (within a defined quality boundary) or has become “out of control.” The risk-adjusted CUSUM analysis is an extension of the original CUSUM method, which plots the difference between the cumulative expected failures (calculated by the Weibull survival model in this study) and the failures that actually occurred. The RA-CUSUM plot gives a visual representation on how far a surgeon's or group of surgeons’ cumulative pouch failure is above or below the predicted cumulative failure, taking into account the expected risk associated with a particular caseload.^12,13 Every case in the series is plotted from left to right on the horizontal axis, and the line moves up for every pouch survival and down for every pouch failure. For each case, the risk of pouch failure is determined by the Weibull survival model, which in turn determines the magnitude by which the graph ascends or descends. For every pouch survival at the time of follow-up, the graph ascends by an amount equal to the estimated probability of pouch failure, and for every pouch failure encountered during follow-up, the graph descends by an amount equal to the estimated probability of pouch survival. Therefore, if a pouch failure occurs in a high-risk patient, the surgeon's performance chart is not unduly penalized. The converse is true if pouch failure occurs in a low-risk patient. In the present study, the RA-CUSSUM graph evaluated the learning curve for IPAA surgery for a number of surgeons, and therefore, each case number represented the aggregated observed minus expected pouch failure for all staff during their first IPAA case, second case, and so on.

Software

The following statistical software package was used: Intercooled STATA 6.0 for Windows (STATA Corporation).

RESULTS

A total of 1965 patients underwent IPAA surgery during the study. The patients’ preoperative characteristics, operative details, and IPAA outcomes are shown in Table 1. The study population was composed of a heterogenous patient population, with histologic diagnoses ranging from 1337 (68.0%) patients with familial adenomatous polyposis or ulcerative colitis, 549 (27.9%) patients with indeterminate colitis to 74 (3.8%) patients with Crohn disease. The median patient follow-up was 4.2 years (ranging between 0 and 19 years). The overall cumulative 5-year (n = 906) and 10-year (n = 372) IPAA survival was 95.7% (95% CI: 94.6–96.8%) and 93.4% (92.2–95.3%).

All procedures were performed by 1 of 12 staff at the Cleveland Clinic Foundation, and these were grouped in accordance to their seniority. The 2 senior staff commenced IPAA surgery in 1983 and 1985, respectively. Of the 10 junior staff, 8 had been trained at the Department of Colorectal Surgery, Cleveland Clinic Foundation, at various time intervals between 1988 and 2001. Each trainee staff had performed on average 67 IPAA procedures during this time period, ranging between 11 and 161 cases. The number of IPAA procedures performed by the senior and junior staff at CCF was grouped by the type of anastomosis and is shown Table 2. The IPAA failure rate is also displayed between the 2 groups of staff. The IPAA failure rate was higher for trainee surgeons in the hand-sewn anastomosis group, a difference which achieved statistical significance (log-rank test for time to failure: 4.66, 1 df, P = 0.031). There was no significant difference in the stapled anastomosis group between senior and junior staff (log-rank test: 0.02, 1 df, P = 0.901).

TABLE 2. Number of Procedures Performed by the Senior (Trainers) and Junior Staff at the Cleveland Clinic Foundation in IPAA surgery

The expected IPAA failure was determined by the multivariate survival model using the following independent predictors of pouch failure: presence of comorbid disease (such as diabetes), prior anal pathology (perianal abscess or fistula), abnormal squeeze pressure and resting pressures on anal manometry, and final diagnosis. The adjusted hazard ratios and their 95% confidence intervals are shown in Table 3. Having adjusted for the type of anastomosis the pouch construction was not found to be an independent predictor of pouch failure; neither was the use of the defunctioning ileostomy. The observed failure rate in the study population was 4.1% (95% CI: 3.5–5.0%) n = 80 and the model predicted failure rate was 3.9%. Causes of failure included pouch-related fistula, n = 17; pelvic sepsis, n = 12; pouch dysfunction, n = 16; chronic pouchitis, n = 11; inflow/outflow obstruction, n = 7; Crohn disease, n = 3; malignancy, n = 3; pouch-related mortality, n = 3; other, n = 8.

TABLE 3. Multivariate Survival Model for Pouch Failure Based on Preoperative Patient Characteristics and Final Histological Diagnosis

The learning curve for all CCF staff (n = 12) for all procedures (stapled and hand-sewn anastomosis) is shown in Figure 1. A visual inspection of the CUSUM plot showed that the IPAA survival was poorer at the beginning of the series and improved after the 49^th consecutive case. To make allowances for the type of anastomosis performed and the prior experience and training circumstances at CCF, risk-adjusted CUSUM charts were contracted for stapled IPAA procedures (Figs. 2 and 3) and hand-sewn IPAA procedures separately (Figs. 4 and 5). As the senior surgeons were considered to be “self-trained,” RA-CUSUM plots were constructed for only trainee staff whose training opportunity commenced in the same institution at various time intervals (Fig. 3 and Fig. 5).

FIGURE 1. Learning curve for IPAA surgery (stapled and handsewn anastomosis) for all staff at CCF (n = 12) comprising senior staff (n = 2), junior staff trained at CCF (n = 8), and junior staff with prior IPAA experience at other institutions (n = 2). A risk-adjusted CUSUM chart for pouch survival is displayed for a series of 1965 consecutive patients undergoing IPAA surgery. The predicted pouch survival was calculated based on a multivariate survival model based on the patient comorbidity, final histology, manometric findings, and the presence of prior anal pathology.

FIGURE 2. Learning curve for senior staff (n = 2) for stapled IPAA surgery.

FIGURE 3. Learning curve for CCF-trained junior staff (n = 8) for stapled IPAA surgery.

FIGURE 4. Learning curve for senior staff (n = 2) for hand-sewn IPAA surgery.

FIGURE 5. Learning curve for CCF-trained junior surgeons (n = 8) for hand-sewn IPAA surgery.

Stapled IPAA Surgery

The learning curve for senior staff is shown in Figure 2. Following a period of 40 cases, there was an improvement in performance in IPAA surgery. Figure 3 displays a RA-CUSUM plot for stapled IPAA anastomosis for the 8 CCF trained junior staff. Four of the 8 trainee surgeons demonstrated a leaning curve in IPAA surgery. Using the aggregated RA-CUSUM plot for all 8 trainee staff, the learning curve for stapled IPAA procedures ended after 23 cases.

Handsewn IPAA Surgery

The learning curve for hand-sewn IPAA cases for senior staff is shown in Figure 4. During the initial training period of 31 cases, a total of 7 pouches failed (3.5 per surgeon), whereas in the subsequent period of 120 consecutive cases, only 4 cases had an associated adverse outcome. The learning curve for hand-sewn IPAA cases for CCF-trained junior staff is shown in Figure 5. There was no demonstrable improvement in pouch outcome at the end of the series of 23 cases, and there was no indication that junior staff learned over their brief series of hand-sewn cases.

Figure 6 shows the ileal-pouch survival curves for the initial training period of 23 cases and beyond for trainee staff; rank-sum test 4.57, P = 0.033. Similar survival curves have been constructed to demonstrate the learning curve for senior staff undertaking IPAA surgery; rank-sum test 3.72, P = 0.054 (Fig. 7).

FIGURE 6. Kaplan-Meier survival curves illustrating the ileal-pouch survival for the initial training period of 23 cases and beyond for trainee staff.

FIGURE 7. Kaplan-Meier survival curves illustrating the ileal-pouch survival for the initial training period of 40 cases and beyond for senior staff.

DISCUSSION

The present study used IPAA failure as a measure of clinical process or task efficiency. IPAA failure is expected to vary between surgeons and is dependent on the differences in case mix, random variation in outcomes, postoperative adverse events, and differences in surgical care. The statistical model provided a measure of surgical performance by making adjustments for the most important patient-dependent risk factors, which included histologic diagnosis, prior anal pathology, abnormal anal manometry, and comorbid disease. Although postoperative events such as anastomotic leakage, pelvic sepsis, anastomotic strictures, and fistula formation play a pivotal role in pouch survival, we elected to omit these from the multivariate model as they describe the process of care and are indirectly related to surgical competency.

What constitutes an acceptable outcome measure as a proxy for clinical effectiveness in IPAA surgery is a contentious issue. Dichotomous rare events such as IPAA survival may be relatively intractable to statistical analysis; however, the present study population was adequate for estimating the regression coefficients for 4 risk factors to support multivariate analysis of such rare events. Other outcomes such as operation time may be relatively easy to collect, but this has been widely criticized as a weak proxy for learning and does not relate to proficiency.^14,15 Outcomes such as postoperative complications (anastomotic leak, pelvic sepsis, or pouch-related fistulae), functional outcomes, and quality of life are alternative measures that could indirectly describe the quality of health care provision in IPAA surgery. In future studies, such outcomes can be used for the characterization of the learning curve in IPAA surgery following appropriate case-mix adjustment.

The RA-CUSUM method consisted of a relatively simple calculation, which can be easily performed on an electronic spreadsheet. Although the RA-CUSUM is not a formal statistical testing procedure, it assists in the process of assimilating complex information on patterns of IPAA survival. The graph provides both numerical and graphical representation of the learning curve in IPAA surgery. Similar to Lovegrove et al,¹³ we have not used any confidence intervals as we have not formally defined what is and is not an acceptable performance in IPAA surgery. In future studies, acceptable failures rates may be established to benchmark individual surgeons’ performance against predefined limits. The risk-adjusted CUSUM model allows weighting of the CUSUM score according to the expected difficulty or risk of the procedure. The Weibull survival model provided the predicted probability of pouch failure, which was then incorporated into the CUSUM calculation. This represents the first attempt in evaluating the learning curve in inflammatory bowel disease surgery using risk-adjusted survival data.

Several technical modifications have changed the original operative technique of the S-pouch configuration with a hand-sewn IPAA.^7,16 Among them was the use of J-pouch reservoir using intraluminal stapling devices for anal pouch anastomosis, with preservation of the anal transition zone.¹⁷ The stapled technique is technically simpler and faster then the alternative hand-sewn technique. It avoids the trauma of mucosectomy by minimizing anal manipulation and provides better manometric and functional results after IPAA as it preserves the anal mucosa just above the dentate line.^18,19 The complexity of hand-sewn anastomosis was reflected in the lack of a true learning curve for senior staff at the clinic. The small number of hand-sewn anastomoses performed by trainee staff over the past decade precluded the characterization of learning in this technique. In contrast, the number of procedures required to attain proficiency in stapled IPAA surgery was readily defined in the study. A longer learning curve was evident for senior staff who were self-trained in IPAA surgery in the early 1980s. A shorter learning curve was evident for stapled IPAA for CCF-trained staff, possibly representing the simplicity and safety of this procedure. Despite the apparent advantages of the stapling technique, there is a need for continuing training of junior staff in hand-sewn anastomosis as the latter technique would be necessary for patients presenting with colorectal cancer, high-grade dysplasia²⁰ in the lower rectum, and for redo ileal pouch procedures.²¹ The learning curve for trainee staff reflects modern developments and training opportunities in IPAA surgery, and these curves possibly set the reference point for subsequent generations of pouch constructors.

The present study reports the training profile of a single major referral center for pelvic pouch surgery. Interpretation of the data should be done with caution, and the results should not be used as a global requirement for all trainee colorectal surgeons. Experience from centers where hand-sewn anastomoses are routinely used may cast light on the learning curve for the hand-sewn technique. The reduction in the steepness of the learning curve poses a challenge to both trainees and trainers. Possible strategies include formal training courses in IPAA surgery, close intraoperative supervision by expert practitioners, and assistance from other well-trained staff.²²

We conclude that the CUSUM method is a useful tool for objective evaluation of practical skills for a group of surgeons during the learning phase of IPAA training. The stapled technique was found to have a shorter learning curve than the hand-sewn technique for IPAA and possibly represents the first choice operation for trainee colorectal surgeons.