Post hoc validation of a tool that accurately predicts the outcome of endoscopic therapy in Bouveret syndrome

Abstract

Background

Bouveret syndrome is characterized by gallstone impaction in the upper gastrointestinal tract causing gastric outlet obstruction. In Bouveret syndrome, endoscopic gallstone removal can avert the need for surgery. However, in cases in which endoscopic therapy is unlikely to succeed, endoscopic attempts delay definitive treatment and compound patient risks. We previously developed a model that predicts endoscopic outcomes from data derived through a systematic review. This tool uses gallstone length, site of impaction, and the number of planned methods of lithotripsy to predict the likelihood of endoscopic success with an accuracy of 81.0%. This study aimed to evaluate our tool performance in an independent, non-training data set and assess endoscopic and surgical outcomes.

Methods

Systematic searches of the PubMed, Scopus, and Cochrane databases were performed for articles published between 16 April 2018 and 1 June 2021. The data reported after our previous study were harvested and inputted into the tool to evaluate their ability to accurately predict outcomes when compared with actual outcomes.

Results

Newly collated data in fields of interest showed no significant statistical differences compared with previous training data sets. Endoscopic therapy was successful in 41.9% of cases. Gallstones of ≤4 cm had a higher chance of successful endoscopic intervention (odds ratio 6.7, 95% confidence interval 1.7–25.8, P < 0.01). Complications of surgery were reported in 29.5%; there was one fatality reported. Post hoc evaluation of our predictive tool demonstrated an AUROC score of 0.80.

Conclusions

We have demonstrated in an independent data set that the tool can be used to accurately predict outcomes of endoscopic therapy. Patients in whom endoscopic therapy is most likely to fail should be offered an early surgical opinion.

Introduction

Bouveret syndrome refers to a rare complication of cholelithiasis, characterized by an impacted gallstone in the stomach or duodenum causing gastric outlet or duodenal obstruction [1]. It classically involves a cholecysto-enteric fistula through which the stone passes. The presentation is often non-specific, with abdominal pain and vomiting the most common presenting symptoms, although hematemesis, weight loss, and anorexia may also occur [1]. Timely diagnosis can be challenging and is typically made using computed tomography (CT), which carries greater diagnostic sensitivity than abdominal ultrasound or esophago-gastro-duodenoscopy [2, 3]. Magnetic resonance imaging may also be helpful, particularly in cases in which gallstones are isoattenuating on CT (15%–25%) [3].

The main goal in managing Bouveret syndrome is to relieve the mechanical obstruction through gallstone fragmentation or extraction. This can be done endoscopically (stone extraction and/or lithotripsy) or surgically (gastrotomy or enterolithotomy). Methods of endoscopic stone fragmentation have evolved with multiple lithotripsy options now available to a wider population of endoscopists: mechanical lithotripsy (ML), extracorporeal shock wave lithotripsy (ESWL), electrohydraulic lithotripsy (EHL), and laser lithotripsy (LL). The management approach taken typically depends on gallstone size, location, the expertise of endoscopists, and equipment availability. Historically, there has been a paucity of guidance for choosing between the two treatment methods (surgery vs endoscopy) and literature has previously recommended endoscopy as the first-line treatment of Bouveret syndrome. This is despite generally poor endoscopic success rates compared with surgery (29% vs 78%) [4, 5]. Failed endoscopic therapy can therefore expose patients to additional risk and lead to a delay in definitive treatment, increasing morbidity and mortality.

Between 2017 and 2018, we conducted an extensive systematic review of Bouveret syndrome and created a novel tool that could accurately predict the likelihood of success of endoscopic therapy in Bouveret syndrome (Figure 1) [5]. Through leave-one-out cross-validation, we demonstrated the tool had an accuracy of 81% within our training data set. However, we have not evaluated its performance in an independent data set of patients, which is important for clinical validation. The primary aim of this study was to generate a new non-training data set through a mini-systematic review and determine whether our tool demonstrated similar performance characteristics compared to our training data set; successful post hoc testing provides further validation of its accuracy for clinical utility. The secondary aims were to further assess outcomes of endoscopic and surgical therapy.

Figure 1.

Tool to predict outcomes of endoscopic therapy in Bouveret syndrome [5]. ML, mechanical lithotripsy; EHL, electrohydraulic lithotripsy; ESWL, extracorporeal shock wave lithotripsy; LL, laser lithotripsy. Credit: This figure has been reproduced with permission [5]. The Creative Commons license does not apply to this content. Use of the material in any format is prohibited without written permission from the publisher, Wolters Kluwer Health, Inc. Please contact permissions@lww.com for further information.

Methods

Selection of studies

We modeled a mini-systematic review against our previously published PROSPERO registered protocol (CRD42018093112) to search all articles published between 16 April 2018 and 1 June 2021. Comprehensive searches of PubMed, Scopus, and Cochrane were completed using the following terms: ‘Bouveret syndrome’, ‘Bouveret’s syndrome’, ‘gallstone’ AND ‘gastric obstruction’, and ‘gallstone’ AND ‘duodenal obstruction’. Access to articles was provided by the UK Access Management Federation via Cambridge University. Articles reporting (i) a confirmed diagnosis of Bouveret syndrome, (ii) the size of the obstructing gallstone, and (iii) the site of stone impaction were included. Our exclusion criteria were as follows: (i) articles not published in English, Spanish, Dutch, French, German, or Greek; (ii) articles not accessible by the UK Access Management Federation; (iii) articles not reporting relevant content; and (iv) articles already used in our previous systematic review. In cases in which the size of the obstructing stone or the location of the obstructing stone was not reported, we attempted to contact the authors of the article to ascertain this information. If this information could not be provided, the cases were excluded from further analysis.

Data extraction

Each identified article was independently reviewed by authors J.O. and C.S. Any disagreements were resolved after discussion with Y.A. Articles that met the inclusion criteria and not the exclusion criteria had data extracted and collated for further analysis. Data recorded were as follows: (i) length of the obstructing gallstone, (ii) site of impaction, (iii) method(s) of endoscopic therapy attempted, (iv) success of endoscopic therapy, (v) whether surgery was performed, (vi) American Society of Anesthesiologists (ASA) grade, and (vii) outcomes of any surgery. The relevant data were then inputted into the predictive tool (Figure 1) to determine the likelihood of success of endoscopic therapy and then this outcome was compared with the patient’s actual outcome. A predicted success rate of >50% was used to define a predicted successful outcome as per our previously described logistic-regression model [5]. Cases of confirmed Bouveret syndrome that did not undergo endoscopic intervention were separately recorded for demographic comparison with our initial data set and further analysis of surgical outcomes.

Risk of bias and quality assessment

As the majority of the articles reviewed were case series and case reports, standard methods such as Egger’s test could not be used to assess for publication bias. To minimize the risk of publication bias and improve the quality of the data-collection process, the authors made attempts to include all articles in the available literature, e.g. multiple attempts at contacting other authors for further or missing information.

Dealing with missing data

Authors of articles with missing data were contacted for more information. If the authors did not respond to our queries, poor-quality studies (insufficient data) were excluded.

Data analyses

MedCalc V.19.1.5 was used for statistical comparisons. Continuous variables (age and gallstone length) were tested for normality using the Shapiro Wilk test. These were found to have non-normal distributions so summary statistics were reported as the median and interquartile range (IQR). For categorical variables (gender, ASA grade, site of gallstone obstruction, endoscopic success rates, and surgical success rates), Chi-square tests were used for statistical comparisons. If n < 5 in any subgroup, the Fisher’s test was used. For continuous variables, the Mann–Whitney U test was used. Reported clinical outcomes were used as a point of reference for the predictive tool. Receiver-operating characteristics (ROC) curve analysis was performed using R to assess the diagnostic performance of the predictive tool against actual clinical outcomes. The area under the ROC (AUROC) score is reported along with its 95% confidence interval (CI).

Results

Description of studies and demographics

A total of 175 articles were identified. Of these, 67 were duplicates and 51 were excluded, leaving 57 articles for further analysis. Among the 57 articles, 38 reported 43 cases in which endoscopic intervention was attempted [6–43] and 19 reported 20 cases of Bouveret syndrome in which no attempt at endoscopic therapy was made and instead first-line surgery was performed [44–62] (Figure 2).

Figure 2.

PRISMA flow diagram of systematic review

Females were affected by Bouveret syndrome more commonly than males (ratio, 1.8:1). The median age of presentation was 78 years (IQR, 66–86 years) and the median ASA grade was 3 (IQR, 2–3). Hereafter, we refer to parts of the duodenum as D1 (the most proximal horizontal part), D2 (the descending part), D3 (the distal horizontal part), and D4 (the distal ascending part). In our study, the most common site of stone impaction was D1 (60.3%) and the median stone size was 43 mm (IQR, 35–50 mm). In all cases identified, an endoscopic intervention was attempted in 68.3% and 31.7% progressed to surgical intervention directly. The demographics of our data set are outlined in Table 1. There were no significant differences in demographic data when comparing our new data set with the training data set from our previous systematic review.

Table 1.

Summary of patient demographics and disease characteristics

Characteristica	New data setb (n = 63)	Old data setc (n = 164)	P-value
Age, years, median (IQR)	78 (66–86)	77 (69–84)	0.85
Gender, n (%)			0.48
Male	21 (36.2)	68 (41.5)
Female	37 (63.8)	96 (58.5)
ASA grade, median (IQR)	3 (2–3)	2 (2–3)	0.50
Gallstone length, mm, median (IQR)	43 (35–50)	45 (40–50)	0.07
Obstruction location, n (%)			0.25
Stomach	9 (14.3)	24 (14.6)
D1	38 (60.3)	77 (47.0)
D2	8 (12.7)	35 (21.3)
D3	8 (12.7)	23 (14.0)
D4	0 (0)	5 (3.0)
Modes of endoscopic therapy, n (%)	n = 42d	n = 100	0.74
Single-mode	18 (41.9)	45 (45.0)
Bi-modal	15 (34.9)	39 (39.0)
Multi-modal	9 (20.9)	16 (16.0)

D1, proximal horizontal part of the duodenum; D2, descending part; D3, distal horizontal part; D4, ascending part; ASA, American Society of Anesthesiologists; IQR, interquartile range.

^a Some cases were excluded from table analysis due to the exact information of patients’ age, gender, and/or ASA grade not being reported.

^b Cases were reported from 16 April 2018 to 1 June 2021.

^c Cases were reported from 1 January 1950 to 15 April 2018.

^d One case was excluded from table analysis due to the exact mode of lithotripsy used not being reported.

Outcomes of endoscopic therapy

Outcomes of endoscopic therapy are summarized in Table 2. Of the 43 cases in which endoscopic therapy was attempted [6–43], 18 were successful (41.9%) [10–12, 16, 17, 19, 20, 22, 24, 29, 31, 33–36, 38]. Gallstones ≤4 cm in size were associated with a higher chance of successful endoscopic intervention than those >4 cm [odds ratio (OR) 6.7, 95% CI 1.7–25.8, P < 0.01]. The median stone size for patients having successful endoscopic therapy was 30 vs 50 mm for those failing endoscopic therapy and requiring second-line surgical intervention (P < 0.01). Gallstones impacted in the proximal upper gastrointestinal tract (stomach/D1/D2) had a 41.7% (15/36) chance of endoscopic success. It was 42.9% (3/7) for gallstones impacted in D3; there were no statistically significant differences on comparison (P = 1.00); 69.8% (30/43) of endoscopic attempts were made in a single endoscopy session, 27.9% (12/43) were made in two sessions, and 2.3% (1/43) were made in four consecutive sessions. It was beyond the scope of this study to evaluate the effectiveness of individual modes of endoscopic therapy (e.g. ML, EHL, LL, and ESWL) and sample sizes in these subgroups were too small for meaningful comparisons. However, a summary of the lithotripsy methods used is provided in Supplementary Table 1. There were insufficient data to evaluate complications from endoscopic therapy since none was reported. Characteristics of successful vs unsuccessful endoscopic cases are shown in Supplementary Table 2.

Table 2.

Success rates in endoscopy and surgery of patients.

Variable	Success rate		P-value
	New data seta	Old data setb
Endoscopic therapy in gallstones ≤4 cm	65.0% (13/20)	66.1% (39/59)	0.93
Endoscopic therapy in gallstones >4 cm	21.7% (5/23)	9.8% (4/41)	0.26
Endoscopic therapy (overall)	41.9% (18/43)	43.0% (43/100)	0.90
Open surgery	96.8% (30/31)	94.0% (109/116)	1.00
Laparoscopic surgery	72.7% (8/11)	50.0% (3/6)	0.60

^a Cases were reported from 16 April 2018 to 1 June 2021.

^b Cases were reported from 1 January 1950 to 15 April 2018.

Outcomes of surgical intervention

Overall, 31.7% (20/63) of cases had surgery performed as first-line treatment [44–58] and 39.7% (25/63) as second-line treatment [6–9, 13–15, 18, 21–23, 25–30, 32, 37, 39–43]. The median stone sizes for patients undergoing first-line surgical therapy and second-line surgical therapy were 40 mm (IQR 30–58 mm) and 50 mm (IQR 40–60 mm), respectively; the difference between both groups was not statistically significant (P = 0.16); 96.8% (30/31) of cases that underwent laparotomy were successful (defined as relief of mechanical obstruction and discharged from hospital), whereas 27.3% (3/11) of laparoscopic cases required conversion to laparotomy. Three cases did not specify whether surgery was via a laparoscopic or open approach [18, 27, 29]. Complications (categorized according to the Clavien Dindo Classification; Supplementary Table 3) were reported in 29.5% (13/44) of surgical cases [6, 8, 18, 22, 41, 43, 44, 46, 47, 50, 59, 61], including one post-operative fatality [61]. Three of the five most serious complications (Clavien Dindo Classification 4a, 4b, or 5) occurred in patients in whom surgery was performed as second-line treatment and endoscopic therapy had failed [6, 22, 41].

Evaluation and post hoc validation of the predictive tool

By defining a >50% chance of endoscopic success as an actual prediction of success, we evaluated the accuracy of the predictive tool in correctly predicting both the successful and failed outcomes of endoscopic therapies by inputting clinical details from references [6] to [43] into the tool. We found that the predictive tool accurately predicted endoscopic outcomes in 35/43 cases (81.4%). The positive likelihood ratio (LR+), negative likelihood ratio (LR−), positive predictive value (PPV), negative predictive value (NPV), sensitivity, specificity, and accuracy are displayed with the ROC curve in Figure 3. Using this new data set, the AUROC score for the tool was 0.80 (95% CI 0.65–0.91).

Figure 3.

Performance data of predictive tool including receiver-operating characteristic (ROC) curve. +LR, positive likelihood ratio; –LR, negative likelihood ratio; PPV, positive predictive value; NPV, negative predictive value.

Discussion

This review found no significant differences in patient age, gender, or ASA grade when compared with our initial systematic review. Importantly, the median stone length was identical in both new and old data sets, and no statistically significant differences were detected when the two populations were compared. Therefore, any differences in the performance of the tool were unlikely to have been influenced by cohort differences. Interestingly, D1 was again the most common site of obstruction.

As would be expected, smaller stones (≤4 cm) were associated with increased success rates of endoscopic therapy. There were insufficient cases to assess whether certain methods of lithotripsy were more effective than others in this data set. It is noteworthy that this predictive tool was modeled on data that used standard gastroscopes to administer endoscopic therapy. Gastroscopes are invariably and ∼1 m in length and this likely limits the effectiveness of endoscopic therapy for gallstones impacted in the distal duodenum. By using far-reaching endoscopes (e.g. enteroscopes or pediatric colonoscopes), lithotripsy and extraction of gallstones can be further optimized and will likely influence the likelihood of success. However, due to the rarity of Bouveret syndrome, studies assessing outcomes between the use of short and long scopes for the treatment of distal duodenal gallstones are currently lacking.

The accuracy of the predictive tool in this data set was 81.4% and this compares to an accuracy of 81.0% in our previous study (81.4% vs 81.0%, P = 0.96). Similarly, the AUROC score was 0.80 (95% CI 0.65–0.91) and it falls within the CI of our previous study (AUROC score = 0.86; 95% CI 0.79–0.94). In terms of performance statistics (as quoted in Figure 3), these were all within the 95% CI of our previous study, which were as follows: new sensitivity 72.2% (previous study 74.4%, 95% CI 58.8%–86.5%), new specificity 88.0% (previous study 86.0%, 95% CI 74.2%–93.7%), new PPV 81.3% (previous study 80.0%, 95% CI 67.3%–88.6%), and new NPV 81.5% (previous study 81.7%, 95% CI 72.6%–88.2%). These further show that there are no significant differences in the performance of our predictive tool in the new data set. The smaller sample size of this data set likely explains the slight differences in AUROC scores and performance statistics.

There were several limitations to this study. First, the data set was small (63 cases in total and only 43 cases had therapeutic endoscopy attempted) and collated from mostly case series and case reports given the rarity of Bouveret syndrome. Second, there is a small risk of publication bias. Third, it is important to recognize that institutional factors such as endoscopist experience, availability of lithotripsy devices, and surgical expertise could have impacted individual case outcomes and management decisions.

Conclusions

We conclude that there is no significant difference in tool performance even when an independent, non-training data set is used. Therefore, this tool can be used with a high degree of accuracy to predict outcomes of endoscopic therapy in Bouveret syndrome. Patients in whom endoscopic therapy is most likely to fail should be offered a surgical opinion early.

Supplementary Data

Supplementary data is available at Gastroenterology Report online.

Authors’ Contributions

C.S. was first reviewer and prepared the manuscript and figures. J.O., the second reviewer, conceived of the study, prepared the manuscript, and performed the statistical analysis. M.Z. assisted with data curation and review of case reports. B.S. reviewed statistical analyses and contributed to the methods section. Y.A. supervised and made critical edits before submission. All authors read and approved the final manuscript.

Funding

None.