“Idealized” vs. “True” learning curves: the case of laparoscopic liver resection

Abstract

Background

Learning curves are believed to resemble an “idealized” model, in which continuous improvement occurs until a plateau is reached. We hypothesized that this “idealized” model would not adequately describe the learning process for a complex surgical technique, specifically laparoscopic liver resection (LLR).

Methods

We analyzed the first 150 LLRs performed by a surgeon with expertise in hepatobiliary/laparoscopic surgery but with no previous LLR experience. We divided the procedures performed in 5 consecutive groups of 30 procedures, then compared groups in terms of complications, operative time, length of stay, and estimated blood loss.

Results

We observed an increase in operative complexity (3.3% major operations in Group 1 vs. 23.3% in Group 5, p = 0.05). Complications decreased from Group 1 to Group 2 (20%–3%), but increased again as more complex procedures were performed (3% in Group 2–13% in Group 5). Similar improvement/regression patterns were observed for operative time and EBL.

Discussion

The “true” learning curve for LLR is more appropriately described as alternating periods of improvement and regression until mastery is achieved. Surgeons should understand the true learning curves of procedures they perform, recognizing and mitigating the increased risk they assume by taking on more complex procedures.

Introduction

The development of laparoscopic surgery in the early 1990's represented a paradigm shift in the field of abdominal surgery. For cholecystectomies, appendectomies, and colectomies it quickly became the standard approach. Laparoscopic liver resection (LLR) was first reported in 1992,¹ however its expansion has been considerably slower when compared to other laparoscopic procedures.²

The slow adoption of LLR can be attributed to, at least in part, the perception that LLR is challenging and a difficult procedure to master. Yet, several studies have demonstrated LLR to be safe.^{3, 4, 5, 6} A growing body of work demonstrates improvements in patient outcomes with LLR when compared to a traditional open resection.^{2, 6, 7, 8} Surgeons routinely performing LLR tend to perform resections of increasing difficulty over time.⁹ In this context, many have become interested in understanding the “learning curve” of LLR.

Learning curves have received increased attention,^{10, 11, 12} as availability of perioperative outcomes data and the emergence of national benchmarking standards have transformed historically subjective approaches to performance assessment, certification, and advancement. Intuitively, many expect learning curves for a wide variety of surgical procedures to approximate the S-shaped (or “idealized”) model (Fig. 1a). However, research across diverse fields suggests that learning progression is more complicated, and may in fact occur in a different ‘shape’^{13, 14, 15} (Fig. 1b).

Figure 1.

“Idealized”(a) vs. “true” learning curve (b)

Few studies have analyzed the learning curve effect in LLR,^{16, 17, 18, 19} the most rigorous of which was limited to data collected during the time when the LLR technique was being developed and standardized.¹⁶ We therefore aimed to analyze the learning curve effect of the current LLR technique. We hypothesized that the idealized learning curve model would be insufficient to capture the complex and evolving inputs of experience and expertise of a surgeon learning this important technique.

Materials and methods

Patients

For this study, we analyzed the complete case series of a surgeon who recently started performing LLR, and for which we had access to the complete operative logs. Retrospective review of the operative database identified the first 150 patients operated on by the same surgeon, all of whom underwent LLR at the Massachusetts General Hospital between March 2007 and April 2015. Clinicopathological data was collected for all of these patients. Data analyzed included: gender, age, BMI, ASA score, indications for LLR, numbers of segments resected, operative time, estimated blood loss (EBL), readmissions within 30 days from operation, and length of stay. Morbidity was evaluated as the presence of Clavien–Dindo grade III–V complications.²⁰ A major operation was defined as a resection of four or more segments.²¹ Operative mortality was defined as death within 90 days from the date of operation. Presence of comorbid conditions was evaluated using the Charlson Comorbidity Index (CCI).²²

Surgical technique

The surgical technique for LLR has been extensively described by others.^{23, 24} Briefly, in our case series, the patients were placed in the supine position. In the majority of the cases, three to five ports were utilized. Of these ports, one or two were 11–15 mm ports, with the remainder being 5 mm ports. Hand-ports were used when a significant inferior vena cava dissection was performed. Pneumoperitoneum was maintained with CO₂, and electronic monitoring was used to keep intra-abdominal pressure between 12 and 15 mmHg. Parenchymal transection was performed using the Sonosurg. The portal pedicles and hepatic veins were transected utilizing hemolock clips or stapled using a vascular stapler (over the years varying staplers have been utilized). The resected specimen was then placed into an EndoCatch bag and removed either by enlarging a 15 mm port, through the hand port, or via a Pfannenstiel incision.

All the procedures were performed by a single operator, who had expertise in laparoscopic surgery and hepato-biliary pancreatic surgery, but with no experience of LLR prior to this series.

Statistical analysis

Statistical analysis was performed using STATA software, version 13.0 (StataCorp LP). Patients were divided into sequential groups of 30 consecutive cases for the purposes of comparison. Unadjusted comparison for categorical variables was performed using the χ² test, and for continuous variables using t-tests. Multivariable regression analyses were performed using multiple logistic or linear regression models to determine the impact of operator experience on each outcome of interest, namely complications (primary outcome), operative time, estimated blood loss (EBL), and length of stay (secondary outcomes), after controlling for age, gender, presence of malignancy, extent of resection (i.e. major vs. minor), ASA score, and Charlson score. Statistical significance was set at a level of p = 0.05.

Results

Descriptive results

For the 150 patients who underwent LLR, the median age was 65 years (mean = 58.7, SD = 16.3), and 64% of the patients were female. The majority of patients (114, or 76%) underwent resection for malignancy, and 26 patients (17.3%) had a major resection of 4 or more segments. The vast majority of the patients included in the study (147, or 98%) had a Charlson Comorbidity Index score of 2, and only 3 patients had a CCI of 1 (Supplementary Table 1). Median operative time was 163 min (mean = 188, SD = 126), and median estimated blood loss was 200 ml (mean = 449 ml, SD = 748). Overall a total of 17 (11.4%) patients had a Clavien–Dindo grade 3–5 surgical complication. The most common complication was bile leak. There was one death within 90 days in the entire series (0.7%) who was in Group 2.

Unadjusted results

Table1 illustrates the differences in pre-operative, intra-operative, and postoperative variables between groups. A significant increase in the percentage of patients who underwent major resections was observed in the study period (from 3.3% in Group 1–23.3% in Group 5, p = 0.05). Complication rate decreased from 20% in patient Group 1–3% in Group 2; this increased again and plateaued at 10–13% in patient groups 3–5 (Fig. 2). Length of stay significantly decreased over time. In Group 1, 24 of 30 patients (79.3%) were hospitalized for more than 3 days; this number decreased to 11/30 (36.7%) in Group 5 (p < 0.01) (Fig. 3a). Operative time and EBL decreased in the study period with a saw-toothed pattern (Fig. 3b).

Table 1.

Pre-operative, intra-operative, and postoperative variables analyzed.

	Patients
All patients (Groups 1–5)		1–30 (Group 1)		31–60 (Group 2)		61–90 (Group 3)		91–120 (Group 4)		121–150 (Group 5)		p
n	%	n	%	n	%	n	%	n	%	n	%
Major	26	17.3	1	3.3	3	10.0	6	20.0	9	30.0	7	23.3	0.05
Malignancy	114	76.0	24	80.0	28	93.3	24	80.0	19	63.3	19	63.3	0.03
ASA score >2	61	40.7	8	5.3	13	8.6	14	9.3	14	9.3	12	8.0	0.46
CCI = 2	147	98.0	29	96.7	30	100.0	29	96.7	30	100.0	29	96.7
EBL (ml)	449.6 ± 748.0		449.4 ± 504.9		845.3 ± 1348.9		359 ± 433.8		421.7 ± 601.4		197.7 ± 276.6
Transfusions intraOP	13	8.7	4	13.3	5	16.6	2	6.6	2	6.6	0	0.0	0.46
Transfusions postOP	10	6.7	3	10.0	4	13.3	2	6.6	1	3.3	0	0.0	0.37
Transfused intra/postoOP	14	9.3	4	13.3	5	16.6	3	10.0	2	6.6	0	0.0	0.43
Op length (min) ± SD	188.3 ± 126.1		180.3 ± 109.6		199.4 ± 130.6		173.8 ± 100.0		226.7 ± 178.4		163.7 ± 97.5
Complications	17	11.3	6	20.0	1	3.3	3	10.0	3	10.0	4	13.3	0.36
LOS ≤3 days	62	41.3	6	20.7	5	8.1	18	60.0	14	48.3	19	63.3	0.00

Figure 2.

Percentage of complications in each patient group

Figure 3.

(a) Percentage of “long” hospitalizations (more than 3 days) in each patient group. (b) Average operative time (blue line, right Y axis) and estimated blood loss (EBL, red line, left Y axis) in each of the five patient groups

Multivariable results

On multivariable analyses, complication rates in Group 4 were statistically significantly lower than those in Group 1 [OR 0.12 (95% CI 0.02–0.91), p = 0.041] and there was a strong overall trend towards decreased complication rates between Group 1 and all subsequent groups (Fig. 4a). Increased operator experience was also associated with significant decreases in overall LOS between Group 1 and all subsequent groups. The “shapes” of these longitudinal trends were irregular, or ‘saw-toothed’ (Fig. 4a–b). In other words, the slopes of odds ratios alternated between positive and negative.

Figure 4.

(a) Multivariable analysis for complications (Group 1 vs. Groups 2–5) in a model including type of resection, age, gender, malignant vs. benign pathology, and ASA score. The vertical bars indicate the confidence interval, while the horizontal marks indicate the odds ratio. (b) Multivariable analysis for length of stay (Group 1 vs. Groups 2–5) in a model including type of resection, age, gender, malignant vs. benign pathology, and ASA score. The vertical bars indicate the confidence interval, while the horizontal marks indicate the odds ratio. *p < 0.05

Discussion

We have analyzed the learning curve effect of the first 150 cases of LLR performed by a surgeon with experience in laparoscopic and hepatic surgery, but with no experience in LLR prior to this series. An overall decrease in complication rates and length of stay was observed. More interestingly, we found that initial periods of improvement are followed by periods of regression, in which LOS and complication rates transiently increase before improving again.

The learning curve for LLR can therefore be said to follow an uneven pattern, with alternating periods of improvement and regression amid an overall trend towards improvement. These results are in contrast with the commonly perceived notion of an “idealized” learning curve, where the slope of learning as expressed against an outcome of interest maintains directionality as it approaches a plateau (Fig. 1a). The plateau represents a time when mastery has been reached by the operator and little to no further improvement can be achieved.¹¹ This “idealized” learning curve model has been studied in other settings; for example, in the manufacturing industry. In one report, assembling times of a Boeing 787 were found to decrease consistently until a plateau phase was reached.²⁵

Yet, this experience may not translate to the learning curve of a professional skill such as the performance of a complex operation. In the industry example above, manufactured pieces and expected tasks have little variation from one performance to the next. In surgery, however, the subsets (i.e. case-mix) of patients undergoing a new procedure by an individual surgeon often evolves over time. Surgeons may perform less complicated cases early in their experience with a new procedure, then gradually perform increasingly complex cases as they accumulate experience. We observed such an evolution in the current study, as evidenced by the significant increase in major resections performed over time. We believe that these changes in case-mix are one of the most likely causes of the cycling learning curve pattern observed in LLR outcomes.

On the other hand, surgeon confidence, or perhaps overconfidence, may represent a more insidious explanation for why we see periods of improvement followed by regression. This behavioral pattern has been shown in other professional disciplines, where attainment of a certain level of proficiency may cause some individuals to relax more easily, or otherwise to attempt increasingly complex maneuvers that carry inherently higher risk profiles. This pattern serves as a reminder to surgeons of the dangers inherent in our work.

Our study is not the first to report that surgical learning curves do not mirror “idealized” learning curves. Moon et al.²⁶ analyzed the learning curves of two surgeons performing laparoscopic gastrectomies, and reported 3 and 4 alternating “phases” for each surgeon, before a plateau phase was reached. Similarly, in a recent study on 1210 pancreaticoduodenectomies throughout the state of California, it has also been reported that the learning curve at the hospital level is not continuous.²⁷

The results of our case series study are strengthened by a 100% capture of consecutive patients undergoing LLR by a surgeon, starting from the very first case performed. In addition, our series was collected during recent years (2007–2015) in which the surgical technique had already been established, therefore changes in equipment or technique should not have impacted the learning curve. Finally, by including Charlson Comorbidity Index scores in our multivariable models, we have theoretically controlled for the independent impact that variations in patient comorbidities may have on the observed learning curve. We acknowledge that our investigation has several limitations. First, this is a single surgeon study, potentially decreasing the generalizability of our findings. Second, our data was collected at a large academic center, and may not reflect the true learning curves of surgeons operating in smaller or non-academic centers. Third, we were unable to measure what contribution, if any, other members of the surgical team (e.g. anesthesiologists, nurses, residents) had on the learning curve of the surgeon in question.

Our findings warrant further investigation in larger multi-surgeon, multi-institution studies. Future research should also attempt to optimize surgeons' learning curves, and attempt to moderate the regression seen after initial learning phases. The use of increasingly sophisticated simulators has been shown to benefit learning of laparoscopic techniques.²⁸ However, given the complexity of liver resection, the role of simulators has to date been limited.²⁸ Large animal models have been proposed for the training in LLR, yet their use is limited by anatomical differences.^{29, 30} Similarly, cadaveric models have been adopted in some settings, but their widespread use in LLR is limited by economic considerations.³¹ Coaching from experts, as well as decision-making tools to facilitate appropriate selection of operative candidates, could also be helpful in augmenting technical training while improving the learning curve.

In summary, the learning curve for LLR is comprised of cycles of alternating improvement and regression. The “true” learning curve is far more complex than what is suggested by the “idealized” model. Surgeons adopting LLR should be aware of their own evolution along the learning curve, and should remain attentive throughout, in particular until a plateau phase is reached. Further studies should be directed towards identifying factors affecting the LLR learning curve, with the goal of improving the learning process and optimizing patients outcomes.

Conflict of interest

None to declare.

Appendix A. Supplementary data

The following is the supplementary data related to this article: