CAGS and ACS Evidence Based Reviews in Surgery. 41

The term “evidence-based medicine” was first coined by Sackett and colleagues as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”¹ The key to practising evidence-based medicine is applying the best current knowledge to decisions in individual patients. Medical knowledge is continually and rapidly expanding. For clinicians to practise evidence-based medicine, they must have the skills to read and interpret the medical literature so that they can determine the validity, reliability, credibility and utility of individual articles. These skills are known as critical appraisal skills, and they require some knowledge of biostatistics, clinical epidemiology, decision analysis and economics, and clinical knowledge.

Evidence Based Reviews in Surgery (EBRS) is a program jointly sponsored by the Canadian Association of General Surgeons (CAGS) and the American College of Surgeons (ACS) and is supported by an educational grant from ETHICON and ETHICON ENDO-SURGERY, both units of Johnson & Johnson Medical Products, a division of Johnson & Johnson and ETHICON Inc. and ETHICON ENDO-SURGERY Inc., divisions of Johnson & Johnson Inc. The primary objective of EBRS is to help practising surgeons improve their critical appraisal skills. During the academic year, 8 clinical articles are chosen for review and discussion. They are selected for their clinical relevance to general surgeons and because they cover a spectrum of issues important to surgeons, including causation or risk factors for disease, natural history or prognosis of disease, how to quantify disease, diagnostic tests, early diagnosis and the effectiveness of treatment. A methodological article guides the reader in critical appraisal of the clinical article. Methodological and clinical reviews of the article are performed by experts in the relevant areas and posted on the EBRS website, where they are archived indefinitely. In addition, a listserv allows participants to discuss the monthly article. Surgeons who participate in the monthly packages can obtain Royal College of Physicians and Surgeons of Canada Maintenance of Certification credits and/or continuing medical education credits for the current article only by reading the monthly articles, participating in the listserv discussion, reading the methodological and clinical reviews and completing the monthly online evaluation and multiple choice questions.

We hope readers will find EBRS useful in improving their critical appraisal skills and in keeping abreast of new developments in general surgery. Four reviews are published in condensed versions in the Canadian Journal of Surgery and 4 are published in the Journal of the American College of Surgeons. For further information about EBRS, please refer to the CAGS or ACS websites. Questions and comments can be directed to the program administrator, Marg McKenzie, at mmckenzie@mtsinai.on.ca.

Selected Article

Wilson E, Gurusamy K, Gluud C, et al. Cost–utility and value-of-information analysis of early versus delayed laparoscopic choleystectomy for acute cholecystitis. Br J Surg 2010;97:210–9.

Abstract

Objective: To compare the cost-effectiveness of early laparoscopic choleystectomy (ELC) versus delayed laparoscopic cholecystectomy (DLC) in patients with acute cholecystitis. Design: A decision-tree model was developed using a series that modelled all potential outcomes for both treatment options. Probabilities were estimated from a Cochrane review. Costs were based on the UK National Schedule of Reference Costs for the year 2006. Setting: UK National Health Service. Patients: Patients with acute cholecystitis. Intervention: Either ELC or DLC with a time frame of 1 year. Main outcome: Outcomes were measured in quality-adjusted life years (QALY) gained over 1 year. Results: Early laparoscopic choleystectomy is less costly and results in better quality of life (+0.05 QALY per patient) than DLC. Given a willingness-to-pay threshold of £20 000 per QALY gained, there is a 70.9% probability that ELC is more cost-effective than DLC. Conclusion: On average, ELC is less expensive and results in better quality of life than DLC.

Commentary

Gallstone-related disease in western populations is one of the most common ailments that general surgeons treat. Between 10% and 15% of the population in western nations have gallstones, and of these, between 1% and 4% will be become symptomatic each year.^1,2 About 30% of cholecystectomies performed are for acute cholecystitis. The advent of laparoscopic cholecystectomy as the preferred method of gallbladder removal seems to have led to reluctance to perform early laparoscopic cholecystectomy (ELC), largely as a result of fear of conversion to open cholecystectomy and the risk of bile duct injury. As a consequence, delayed laparoscopic cholecystectomy (DLC), which is performed about 6 weeks later than ELC, has gained popularity, probably owing in part to unfamiliarity with open cholecystectomy among surgeons who have trained in the last 15–20 years. However, a number of studies have demonstrated the safety of ELC. A potential drawback of the DLC approach includes recurrent symptoms in the weeks leading up to the procedure that may necessitate treatment. This may result in high conversion rates, longer total hospital stay and greater loss of productivity owing to more time away from work. Studies comparing DLC and ELC have established the clinical effectiveness of the ELC approach.^1–3 Thus, given the large number of cholecystectomies performed for acute cholecystitis and the ongoing popularity of the DLC approach, this study by Wilson and colleagues⁴ is very relevant. Importantly, it approaches the issue from a different perspective: cost-effectiveness.

The authors used a transparent and reproducible process. First, using a decision tree and source data from a previous meta-analysis² comparing the effectiveness of the 2 options, the authors calculated the average cost for a patient in each treatment group. Examination of this tree reveals it to be comprehensive and considers all the possible clinical outcomes (excluding the no surgery/percutaneous cholecystostomy conservative approach in poor surgical candidates). The probabilities used in the tree have largely been drawn from a Cochrane review and meta-analysis that collated data from 5 randomized controlled trials (RCTs) comparing ELC and DLC.¹ The costing data were derived from the UK National Health Service (2006). Second, the authors calculated a composite measure of both quality and length of life (quality-adjusted life years, QALY) for patients in each treatment group. The analysis compared the economic implications for both strategies, including all potential clinical scenarios encountered when managing these patients with either approach. Third, the authors determined cost-effectiveness by calculating a ratio of the difference in costs and QALY between the 2 treatment options. By comparing this ratio with the maximum willingness-to-pay threshold, they determined cost-effectiveness.

One of the main issues needing critical evaluation is the costing data used in this study. Costs were based on the published values in the UK National Schedule of Reference Costs for the year 2006. These values are specific to the United Kingdom. The authors list all costs in an appendix to the article and describe any variations in cost that they used. This is important, as it allows others to plug in costing information specific to their own environment, if available. Readers can rerun the analysis with costs specific to their health care system. Although the main limitation of this study was the external validity of the cost estimates, which would likely be different in the United States and Canada, the relative amount and frequency of various charges would likely be consistent among different types of payer systems. Whereas the absolute values would be different, the conclusion would likely be the same in the American and Canadian health care systems.

The other area that needs critical review is the source data used to construct the decision tree. These data were derived from 5 RCTs collated in the Cochrane review and meta-analysis.¹ A critical component that can affect complication and conversion rates is the surgeon’s experience. In the 5 RCTs, the experience of the surgeons was variable; hence valid conclusions could not be drawn with regard to the surgeons’ experience and the outcomes. However, it is encouraging to note that in 2 of the RCTs, the surgeons’ experience seemed to be limited. This point is perhaps the most relevant, as it may be lack of experience, lack of comfort with laparoscopic cholecystectomy to treat acute cholecystitis or fear of conversion to the open procedure that prevents more surgeons from attempting ELC. The Cochrane review and meta-analysis does not provide evidence to encourage a change in practice. In fact, the authors state that “ELC should only be performed by surgeons with adequate laparoscopic experience and prior experience of operating during the acute cholecystitis.”

The values derived from the meta-analysis² did in some cases have wide confidence intervals. The quality of the individual studies used in the meta-analysis was not critically reviewed by Wilson and colleagues;⁴ however, these are the best studies available from which to obtain and construct the values used in the decision tree. To examine the robustness of the findings, the authors used a probabilistic sensitivity analysis in which they chose a random value from the corresponding distribution for each cost, outcome and probability at each node in the decision tree. This generated an estimation of the cost and QALY gained from each strategy. Through this analysis, the authors showed that there is strong evidence that ELC is cheaper than DLC (almost all scatter points in the negative half of the incremental cost axis); however, there is still some uncertainty as to which approach results in better quality of life (large concentration of scatter points in the negative half of the QALY axis).

The only other major issue that potentially needs clarification relates to the definitions of early and delayed laparoscopic cholecystectomy. Is there a time point at which ELC should be performed and after which conversion rates and complications significantly rise? It is possible that refinements in these definitions may result in better outcomes. Future studies may be warranted to define the safest time interval after an episode of acute cholecystitis in which to remove a gallbladder. This, however, does not change the findings of the study by Wilson and colleagues,⁴ nor the importance of those findings.

Although other studies have assessed the safety and effectiveness of ELC for acute cholecystitis, no other study so thoroughly analyzes the cost-effectiveness (efficiency) of ELC. Wilson and colleagues⁴ make a strong case that ELC should be the treatment of choice for acute cholecystitis, not only because it may be safer, but also because it is more cost-effective and results in a better quality of life than DLC. The combination of the results of this study with those of previous meta-analyses and other clinical studies provide compelling evidence that, if they are not already doing so, surgeons should perform ELC for most patients with acute cholecystitis. The cost and quality of life data were derived from high-quality sources, and given the available data, the evidence supports the authors’ conclusion. It is up to each surgeon to determine his or her level of comfort with ELC. Those of us who perform ELC routinely should continue to do so, with appropriate monitoring of outcomes. Those of us who lack comfort or experience with ELC in this clinical setting should change our practices only in the context of appropriate mentoring.