Abstract
This study examines outcomes of laparoscopic cholecystectomies performed by surgical residents before and after mandatory implementation of Fundamentals of Laparoscopic Surgery certification to assess whether certification is associated with residence performance or patient outcome.
The Fundamentals of Laparoscopic Surgery (FLS) program was developed to assess the skills and knowledge essential to surgeons as laparoscopy emerged, with the goal of improving quality of care and safety, including reducing complication rates.1 As laparoscopy has become mainstream, the ongoing value of FLS certification as a high-stakes examination comes into question. We hypothesized that implementing mandatory FLS certification would not appreciably alter resident performance of or patient outcomes after laparoscopic cholecystectomy (LC).
Methods
A retrospective review of all LCs (urgent, emergent, and elective) performed by junior surgical residents (postgraduate year [PGY]1-3) with a senior resident (PGY4-5) teaching assistant at a university-affiliated public teaching hospital in Torrance, California, was completed for 2 periods: before mandatory FLS certification (2005-2009) and after mandatory FLS certification (2010-2014). An FLS certification at our institution is obtained in the research year between PGY3 and PGY4. Outcome measures comprised a composite of all intraoperative complications, including intraoperative bile duct injury (BDI), surgery length, need for conversion to open cholecystectomy or biliary bypass, overall hospital complications, length of stay, and 30-day readmission. The Los Angeles Biomedical Research Institute Institutional Review Board approved this study and waived the need for obtaining patient informed consent given the minimal risk to patients involved in this study.
The before and after FLS certification categorical outcomes were compared using 2-tailed χ2 or Fisher exact tests, whereas the continuous variables were analyzed with the Wilcoxon rank sum test. For significantly associated variables in bivariate analyses, multivariable regression analyses were performed. Statistical analyses were conducted from November 1, 2015, to September 30, 2017, using SAS, version 9.3 (SAS Institute Inc). A 2-sided P < .05 was considered to be statistically significant.
Results
During the study period, 3034 LCs were performed. Acute cholecystitis (1126, 37.1%) and symptomatic cholelithiasis (1019, 33.6%) were the most common preoperative diagnoses. Overall, there were 35 intraoperative complications (1.2%), with 10 BDIs (0.3%) (Table 1). The results of bivariate analyses comparing data before and after the FLS certification requirement indicated no significant difference with respect to intraoperative complications (odds ratio [OR], 1.02; 95% CI, 0.53-1.97; P = .95), BDI (OR, 0.52; 95% CI, 0.15-1.84; P = .53), need for conversion to open surgery (OR, 0.80; 95% CI, 0.61-1.05; P = .10) or biliary bypass (OR, 2.43; 95% CI, 0.22-26.81; P = .45), or surgery length (OR, 0.30; 95% CI, 0-0.50; P = .06); however, overall hospital complications (OR, 1.59; 95% CI, 1.01-2.47; P = .04), hospital length of stay (OR, 1.0; 95% CI, 0-2.0; P = .001), and 30-day readmissions (OR, 7.83; 95% CI, 3.99-15.33; P < .001 were significantly different (Table 2). The results of multivariable analyses indicated that patient age was the only factor independently associated with intraoperative complications (OR, 1.03; 95% CI, 1.01-1.06; P = .006); no factor was associated with longer length of stay, and both increasing patient age (OR, 1.03; 95% CI, 1.01-1.04; P < .001) and after mandatory FLS certification (OR, 7.6; 95% CI, 3.8-15.0; P < .001) were significantly associated with readmission.
Table 1. Demographic and Clinical Characteristics and LC Outcomes Among 3034 Patients (2005-2009 and 2010-2014).
| Characteristic or Outcome | Patients, No. (%) |
|---|---|
| Age, median (IQR), y | 38 (28-51) |
| Sex | |
| Male | 623 (20.5) |
| Female | 2411 (79.5) |
| Diabetes | 183 (6.0) |
| Preoperative diagnosisa | |
| Acute cholecystitis | 1126 (37.1) |
| Symptomatic cholelithiasis | 1019 (33.6) |
| Gallstone pancreatitis | 441 (14.5) |
| Choledocholithiasis | 421 (13.9) |
| Chronic cholecystitis | 69 (2.3) |
| Cholangitis | 48 (1.6) |
| Other | 44 (1.5) |
| Intraoperative cholangiogram | 1173 (38.7) |
| Intraoperative complication | 35 (1.2) |
| Bile duct injury | 10 (0.3) |
| CHD | 4 (0.1) |
| CBD | 4 (0.1) |
| Combined CBD and CHD | 2 (0.1) |
| Bleeding | 8 (0.3) |
| Right hepatic artery injury | 2 (0.1) |
| Liver laceration | 5 (0.2) |
| Serosal colon injury | 1 (0.03) |
| Veress needle injury | 2 (0.1) |
| Cystic duct stump leak | 7 (0.2) |
| Surgery length, median (IQR), h | 1.6 (1.2-2.2) |
| Conversion to open cholecystectomy | 232 (7.6) |
| Biliary bypass performed | 3 (0.1) |
| Overall hospital complicationsb | 80 (2.6) |
| Hospital length of stay, median (IQR), d | 4 (1-6) |
| Mortality | 0 |
| 30-d Readmissions | 72 (2.4) |
Abbreviations: CBD, common bile duct; CHD, common hepatic duct; IQR, interquartile range; LC, laparoscopic cholecystectomy.
Some cases were coded with multiple preoperative diagnoses.
Includes cerebrovascular accident, myocardial infarction, pulmonary embolism, deep vein thrombosis, pneumonia, respiratory failure, bacteremia, colitis, urinary retention, urinary tract infection, postoperative bleeding, retained common bile duct stone, or pancreatitis.
Table 2. Comparison of Baseline Demographic Characteristics and Outcomes Before and After FLS Certification Required.
| Characteristic or Outcome | Patients, No. (%)a | OR (95% CI) | P Value | |
|---|---|---|---|---|
| Before FLS | After FLS | |||
| Total No. of patients | 1663 (54.8) | 1371 (45.2) | NA | NA |
| Age, median (IQR), y | 37 (28-49) | 40 (29-53) | 3 (1-4)b | <.001 |
| Female | 1346 (80.9) | 1065 (77.7) | 0.82 (0.69-0.98) | .03 |
| Diabetes | 68 (4.1) | 115 (8.4) | 2.15 (1.58-2.92) | <.001 |
| Preoperative diagnosisc | ||||
| Acute cholecystitis | 535 (32.2) | 591 (43.1) | 1.60 (1.38-1.85) | <.001 |
| Symptomatic cholelithiasis | 668 (40.2) | 351 (25.6) | 0.51 (0.44-0.60) | <.001 |
| Gallstone pancreatitis | 242 (14.6) | 199 (14.5) | 0.99 (0.81-1.22) | .98 |
| Choledocholithiasis | 173 (10.4) | 248 (18.1) | 1.90 (1.54-2.34) | <.001 |
| Chronic cholecystitis | 17 (1.0) | 52 (3.8) | 3.81 (2.20-6.63) | <.001 |
| Cholangitis | 22 (1.3) | 26 (1.9) | 1.44 (0.81-2.56) | .21 |
| Other | 5 (0.3) | 39 (2.8) | 9.71 (3.82-24.70) | <.001 |
| Intraoperative cholangiogram | 722 (43.4) | 451 (32.9) | 0.64 (0.55-0.74) | <.001 |
| Intraoperative complication | 19 (1.1) | 16 (1.2) | 1.02 (0.53-1.97) | .95 |
| Bile duct injury | 7 (0.4) | 3 (0.2) | 0.52 (0.15-1.84) | .53 |
| Surgery length, median (IQR), h | 1.55 (1.11-2.12) | 1.60 (1.17-2.17) | 0.30 (0-0.50)b | .06 |
| Conversion to open cholecystectomy | 139 (8.4) | 93 (6.8) | 0.80 (0.61-1.05) | .10 |
| Biliary bypass performed | 1 (0.1) | 2 (0.2) | 2.43 (0.22-26.81) | .45 |
| Overall hospital complication | 35 (2.1) | 45 (3.3) | 1.59 (1.01-2.47) | .04 |
| Hospital length of stay, median (IQR), d | 3 (0-6) | 4 (2-6) | 1 (0-2)b | .001 |
| Mortality | 0 | 0 | NA | NA |
| 30-d Readmission | 10 (0.6) | 62 (4.5) | 7.83 (3.99-15.33) | <.001 |
Abbreviations: FLS, Fundamentals of Laparoscopic Surgery; IQR, interquartile range; NA, not applicable; OR, odds ratio.
Categorical variables expressed as No. (%); continuous variables, as median (IQR).
Median difference (95% CI).
Some cases were coded with multiple preoperative diagnoses.
Discussion
In this study, after implementation of mandatory FLS certification, we found no appreciable improvement in outcomes of LCs performed by residents as measured by several outcome measures, including intraoperative complications, surgery length, and the need for conversion to open cholecystectomy.
A recent systematic review2 finding limited data to support the validity of FLS manual skill examination tasks or the scoring method suggested that demonstrating differences in scores between novices and experts does little to confirm content validity. In a review of 53 632 LCs from an insurance database, Schwaitzberg et al3 reported that FLS-certified surgeons had, counterintuitively, a higher rate of BDI compared with non-FLS−certified surgeons (0.47% vs 0.14%, P = .001). A 2003 study4 determined that 97% of BDIs during LC were attributable to errors in perception, judgment, and knowledge, but the FLS examination addresses only basic laparoscopic skills.1 When combined with the results of the present study, those study results bring into question whether FLS certification positively influences rates of intraoperative complications, one of the stated certification goals.
This study is limited by its retrospective design and potential selection bias given that it is a single-institution study. In addition, the residents at our institution participate very early in hands-on training in laparoscopic surgery; thus, it is possible that owing to this extensive early exposure, the operative outcomes at our institution were not substantially influenced by the implementation of mandatory FLS certification.
Today’s residents face greater hurdles to achieve board certification, including mounting student debt, longer training periods, and a growing list of mandatory certifications.5 The first-time pass rate for FLS is 96%,6 suggesting a low discriminating value. There does not appear to be evidence that mandatory FLS certification has improved LC outcomes. As such, one must reassess the value of continuing FLS in its current format as a high-stakes examination.
References
- 1.Fundamentals of Laparoscopic Surgery. Fundamentals of Laparoscopic Surgery program description. https://www.flsprogram.org/index/fls-program-description/. Accessed December 7, 2017.
- 2.Zendejas B, Ruparel RK, Cook DA. Validity evidence for the Fundamentals of Laparoscopic Surgery (FLS) program as an assessment tool: a systematic review. Surg Endosc. 2016;30(2):512-520. doi: 10.1007/s00464-015-4233-7 [DOI] [PubMed] [Google Scholar]
- 3.Schwaitzberg SD, Scott DJ, Jones DB, et al. Threefold increased bile duct injury rate is associated with less surgeon experience in an insurance claims database: more rigorous training in biliary surgery may be needed. Surg Endosc. 2014;28(11):3068-3073. doi: 10.1007/s00464-014-3580-0 [DOI] [PubMed] [Google Scholar]
- 4.Way LW, Stewart L, Gantert W, et al. Causes and prevention of laparoscopic bile duct injuries: analysis of 252 cases from a human factors and cognitive psychology perspective. Ann Surg. 2003;237(4):460-469. doi: 10.1097/01.SLA.0000060680.92690.E9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.The American Board of Surgery. Training requirements. http://www.absurgery.org/default.jsp?certgsqe_training. Published 2017. Accessed January 19, 2018.
- 6.Bilgic E, Kaneva P, Okrainec A, Ritter EM, Schwaitzberg SD, Vassiliou MC. Trends in the Fundamentals of Laparoscopic Surgery (FLS) certification exam over the past 9 years. Surg Endosc. 2018;32(4):2101-2105. doi: 10.1007/s00464-017-5907-0 [DOI] [PubMed] [Google Scholar]
