Skip to main content
Annals of Surgery Open logoLink to Annals of Surgery Open
. 2021 Sep 14;2(3):e097. doi: 10.1097/AS9.0000000000000097

Laparoscopic Versus Open Colorectal Surgery in the Emergency Setting

A Systematic Review and Meta-analysis

Anne-Loes K Warps *,†,, Emma S Zwanenburg *, Jan Willem T Dekker §, Rob AEM Tollenaar †,, Willem A Bemelman *, Roel Hompes *, Pieter J Tanis ‖,, Elisabeth J de Groof *
PMCID: PMC10455067  PMID: 37635817

Abstract

Objective:

This systematic review and meta-analysis aimed to compare published outcomes of patients undergoing laparoscopic versus open emergency colorectal surgery, with mortality as primary outcome.

Background:

In contrast to the elective setting, the value of laparoscopic emergency colorectal surgery remains unclear.

Methods:

PubMed, Embase, the Cochrane Library, and CINAHL were searched until January 6, 2021. Only comparative studies were included. Meta-analyses were performed using a random-effect model. The Cochrane Risk of Bias Tool and the Newcastle-Ottawa Scale were used for quality assessment.

Results:

Overall, 28 observational studies and 1 randomized controlled trial were included, comprising 7865 laparoscopy patients and 55,862 open surgery patients. Quality assessment revealed ‘good quality’ in 16 of 28 observational studies, and low to intermediate risk of bias for the randomized trial. Laparoscopy was associated with significantly lower postoperative mortality compared to open surgery (odds ratio [OR] 0.44; 95% confidence interval [CI], 0.35–0.54). Laparoscopy resulted in significantly less postoperative overall morbidity (OR, 0.53; 95% CI, 0.43–0.65), wound infection (OR, 0.63; 95% CI, 0.45–0.88), wound dehiscence (OR, 0.37; 95% CI, 0.18–0.77), ileus (OR, 0.68; 95% CI 0.51–0.91), pulmonary (OR, 0.43; 95% CI, 0.24–0.78) and cardiac complications (OR, 0.56; 95% CI, 0.35–0.90), and shorter length of stay. No meta-analyses were performed for long-term outcomes due to scarcity of data.

Conclusions:

The systematic review and meta-analysis suggest a benefit of laparoscopy for emergency colorectal surgery, with a lower risk of postoperative mortality and morbidity. However, the almost exclusive use of retrospective observational study designs with inherent biases should be taken into account.


Mini-Abstract: This systematic review and meta-analysis aimed to compare published outcomes of emergency laparoscopic and open colorectal surgery. The pooled results of 1 feasibility randomized controlled trial and 28 observational comparative studies, showed lower postoperative mortality and postoperative morbidity rates following laparoscopy when compared to open surgery.

Supplemental Digital Content is available in the text.

INTRODUCTION

During the past decades, there has been a shift towards laparoscopy in elective colorectal surgery for both benign and malignant colorectal disease. Multiple large randomized studies showed many benefits of laparoscopy compared to open colorectal surgery such as equal safety, reduced pain, less postoperative morbidity, less intensive care unit (ICU) admissions, shorter length of stay (LOS), and lowered costs, as well as increased quality of life.16 Although the use of laparoscopy in the acute setting has also been proven safe and feasible, there is a relatively low uptake of the use of laparoscopy in emergency colorectal procedures. This is probably due to the increased complexity of patients’ presentation, advanced or complicated disease, as well as the experience of the surgeon on call during evening and night-time hours.

Emergency surgery is generally associated with increased morbidity and mortality when compared to the elective setting.7,8 It is hypothesized that the well-established benefits of laparoscopy in the elective setting could also be applicable to the emergency setting. One might even suggest that reducing surgical trauma and stress response, as well as preservation of the abdominal wall integrity, is particularly beneficial for these patients at the highest operative risk. Therefore, laparoscopy might have an impact on one of the most relevant endpoints after colorectal surgery, namely postoperative mortality.

There is a need to elucidate the role of laparoscopy in emergency colorectal surgery, thereby shifting from traditional endpoints in the elective setting such as length of hospital stay to endpoints with the highest clinical impact including postoperative mortality. Prior reviews of the available literature did not perform meta-analyses, and mortality was not included as the main outcome parameter.9,10 Additionally, published reviews are restricted to literature before 2016. Ever since, more relevant studies have been published, including a randomized controlled trial (RCT), and implementation of laparoscopy progressed over time with broadening of the indications.

The aim of this study was to provide an up-to-date systematic review of the current literature on patients with acute colorectal disease who underwent emergency surgery by a laparoscopic versus open approach with meta-analyses of postoperative outcomes, including mortality and morbidity, as reported by comparative studies. In addition, the intention was to assess long-term outcomes.

METHODS

The study protocol was prospectively registered at PROSPERO (registration number: CRD42020189955), and the Preferred Reporting Items for Systematic Reviews and Meta-analysis guidance was followed throughout the process.11 Study selection, data extraction, and quality assessment were independently performed by two reviewers (A.K.W. and E.S.Z.). Any disagreements were resolved through discussion until consensus was reached or discussed with a third and fourth reviewer (E.J.d.G. and P.J.T.).

Search

A clinical librarian was consulted for assembling the search strategy for multiple databases, including PubMed, EMBASE, the Cochrane Library, and CINAHL. The search strategy included (Medical Subject Headings) terms and free text related to or describing colorectal disease, emergency setting, laparoscopic, and open surgery. The complete search strategy is presented in Supplementary Table 1, http://links.lww.com/AOSO/A65. The search was performed on June 4, 2020 and updated on January 6, 2021. In addition, reference lists and bibliographies of included studies were hand searched for relevant studies and trial and study registries were searched for relevant ongoing studies (Netherlands Trial Register12 and trails.gov13).

Study Selection

Both randomized and nonrandomized studies comparing outcomes of patients that underwent laparoscopic versus open colorectal surgery for benign or malignant indications in the emergency setting were included. Emergency setting was defined as any nonelective procedure, ranging from acute semielective procedures to urgent emergency procedures. The used definitions are reported in Supplementary Table 2, http://links.lww.com/AOSO/A65. Study designs other than comparative (ie, case series) or studies including noncolorectal procedures (ie, appendectomy, small bowel surgery, or gastro-duodenal procedures) were excluded. Studies reporting on multiple indications for emergency surgery, but providing separate data for the colorectal subgroup were included. Similarly, studies primarily reporting on the elective setting were also included if providing subanalyses of emergency patients. Studies on peritoneal lavage alone or diagnostic laparoscopies were excluded as well as review articles, studies with no full text available, and studies with less than 10 patients. There were no language restrictions, all non-English studies were translated. Excluded studies were listed with the reason for the omission, and references are reported in Supplementary File 1, http://links.lww.com/AOSO/A65.

Quality Assessment

Two reviewers (A.K.W. and E.S.Z.) independently assessed the quality of the included studies. The Cochrane Collaboration Tool for Risk of Bias for randomized studies and the Newcastle-Ottawa scale (NOS) for nonrandomized studies were used.14 The follow-up was scored with one star when the duration was at least 30 days. If follow-up time was not explicitly mentioned, the outcome domain was scored with zero stars. The quality level was then converted to the Agency for Healthcare Research and Quality (AHRQ) standard of “good,” “fair,” and “poor” quality. Thresholds for converting the NOSs to AHRQ standards (good, fair, and poor)15 were conducted as described by several previous systematic reviews.1618

No funnel plots were assessed for publication bias since dichotomous outcomes with intervention effects expressed as odds ratio (OR), the standard error of the log OR is mathematically linked to the size of the OR, even in the absence of small-study effects causes unreliable funnel plots.19,20

Outcomes and Definitions

The primary outcome was postoperative mortality, which was defined as any deaths reported in the postoperative course regardless of follow-up time. The secondary outcome was overall postoperative morbidity, which was defined as the total complication rate. Other secondary outcomes were ICU admission, reinterventions, wound infection, wound dehiscence, postoperative ileus, anastomotic leakage, intra-abdominal infection or abscess, pulmonary complications, cardiac complications, LOS, readmissions, and long-term outcomes. Furthermore, conversion to open surgery in the laparoscopic group was reported. The precise definitions of the outcomes as reported in the original studies are presented in Supplementary Table 5, http://links.lww.com/AOSO/A65. Long-term outcomes included incisional hernias and oncological outcomes such as survival or recurrence rates.

Data Extraction

A predefined data extraction table was used for data collection and included general study characteristics, patients’ characteristics, indications for surgery and procedural characteristics, predefined outcomes, used definitions, and quality assessment. Data were extracted from the included studies if the authors reported our predefined outcome measures or any exact synonyms. Postoperative overall morbidity rates were not calculated if not reported by the authors to prevent overlapping complications within patients. Pulmonary and cardiac complications were composite endpoints, thereby including all reported pulmonary (ie, pneumonia and pulmonary insufficiency) and cardiac complications (ie, arrhythmias and myocardial infarction). If there were data from propensity score analyses or intention-to-treat analyses, the results from these analyses were used rather than the original data.

Data Synthesis and Statistical Analysis

The primary outcome and secondary outcomes were quantitatively summarized. Review Manager version 5.4.0 (The Cochrane Collaboration, London, United Kingdom, 2020) was used for performing meta-analyses. For dichotomous variables, the pooled effect was analyzed using a Mantel-Heanzel test and a random effect model. The pooled effect was reported as OR with corresponding 95% confidence intervals (CIs). Continuous data were analyzed by an inverse-variance method static with a random effect model and expressed as the standardized mean differences (SMDs) with corresponding 95% CI. To calculate the SMD, only studies that reported the mean and SD could be included in the meta-analysis. Therefore, studies reporting medians and interquartile ranges (IQRs) or medians and ranges (minimum and maximum) were converted to estimated means and estimated SDs using the method described by Wan et al21 Subgroup analyses for morbidity stratified by indication for surgery (eg, colorectal cancer, diverticulitis and inflammatory bowel disease [IBD]) were performed. Heterogeneity between studies was perceived as considerable when I2 > 75%. A P value of <0.05 was considered statistically significant. If there were zero events of the outcome measured in both arms of the study population, the study was excluded from the meta-analysis.22 A sensitivity analysis was performed for the primary outcome by evaluating the change in pooled OR when using a fixed-effect model instead of a random effect model, when including solely observational or RCTs, and when including only those studies in the meta-analysis that scored “good” quality on the NOS. There were no deviations from the proposed protocol during study selection, data extraction, statistical analysis, or quality assessment.

RESULTS

Included Studies

The initial literature search identified a total of 2841 studies. One additional study was found by hand-searching reference lists. After the removal of duplicates, 1773 studies remained for title and abstract screening. Eventually, 61 studies were assessed for eligibility based on full text, of which 29 studies were eligible, including 1 feasibility RCT and 28 retrospective cohort studies, including several large population-based studies (Fig. 1). One study was funded by a company that financially supported the author,23 five studies received financial support or grants from the government, healthcare improvement programs, or foundations but declared no (financial) conflicts of interest,2428 and seven studies provided no information on conflicts of interests or financial support.2936 Characteristics of the included studies are presented in Table 1.

FIGURE 1.

FIGURE 1.

PRISMA flowchart of the included studies in this systematic review and meta-analysis of the primary outcome (ie, postoperative mortality; last search January 6, 2021). Reasons for exclusion after full-text reading: wrong study population (nonemergency patients and noncolorectal patients), wrong intervention (intervention other than laparoscopy or open surgery), wrong study design (noncomparative studies, case-series, systematic reviews, and conference abstracts). PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-analysis.

TABLE 1.

Study Characteristics and Quality Assessment

Study Year Country Study Design Period Centre(s) N Quality Assessment*
Ballian et al 2012 USA RC 2005–2008 MC 3,552 8 out of 9
Cassini et al 2017 Italy RC 2008–2016 SC 60 7 out of 9
Catani et al 2011 Italy RC 2007–2009 SC 93 6 out of 9
Cocorullo et al 2016 Italy RC 2013–2014 SC 159 4 out of 9
Cui et al 2019 China RC 2013–2018 SC 128 7 out of 9
Dunker et al 2000 NL RC 1996–1999 MC 42 5 out of 9
Gietelink et al 2016 NL RC 2010–2013 MC 5,192 7 out of 9
Harji et al 2020 UK RCT 2016–2017 MC 64 NA
Keller et al 2016 USA RC 2008–2011 MC 22,719 7 out of 9
Kim et al 2015 Korea RC 2008–2013 SC 49 7 out of 9
Koh et al 2013 Singapore RC 2006–2011 SC 46 6 out of 9
Lee et al 2019 USA RC 2012–2017 MC 3,756 8 out of 9
Letarte et al 2015 Canada RC 2000–2010 SC 125 5 out of 9
Li et al 2009 China RC 2001–2006 SC 18 5 out of 9
Li et al 2015 China RC 2013–2013 SC 35 5 out of 9
Liu et a 2014 China RC 2007–2012 SC 193 7 out of 9
Marceau et al 2007 France RC 1999–2006 SC 88 7 out of 9
Marcello et al 2011 USA RC 1997–1999 MC 48 5 out of 9
Moghadamyeghaneh et al 2020 USA RC 2012–2017 MC 1,293 8 out of 9
Nash et al 2010 USA RC 2001–2006 SC 68 6 out of 9
Ng et al 2008 China RC 2003–2006 SC 43 6 out of 9
Odermatt et al 2013 UK RC 2006–2011 SC 108 7 out of 9
Schloricke et al 2013 Germany RC 1997–2009 SC 36 4 out of 9
Stulberg et al 2009 USA RC 2005–2008 SC 67 8 out of 9
Sujatha-Bhaskar et al 2017 USA RC 2012–2014 MC 10,018 8 out of 9
Turley et al 2013 USA RC 2005–2009 MC 134 8 out of 9
Vallance et al 2019 UK RC 2010–2016 MC 15,516 8 out of 9
Vennix et al 2015 NL RC 2010–2014 MC 117 7 out of 9
Watanabe et al 2009 Japan RC 2000–2004 SC 60 5 out of 9

Study characteristics and Newcastle-Ottawa Scale score (NOS score).

MC, multicenter; RC, retrospective cohort; RCT, randomized controlled trial; SC, single center.

Quality Assessment

The individual scores of the 28 retrospective cohort studies varied from 4 to 8 out of 9 on the NOS (Table 1). When converting the NOS rating to the AHRQ, standards, 16 of the 28 retrospective cohort studies showed to be of “good” quality, whereas one study was scored to have “fair” quality and 11 studies where scored as “poor” quality. The feasibility RCT by Harji et al24 showed to have a low to moderate risk of bias according to the Cochrane Risk of Bias Tool.

Study Population

In total, 7865 patients underwent emergency laparoscopy and 55,862 patients laparotomy. The main indications for emergency colorectal surgery were perforated or obstructive colorectal cancer, complicated diverticulitis, and severe, therapy refractory IBD. Patient characteristics and indications for emergency colorectal surgery, as well as procedural characteristics of the included studies, are shown in Supplementary Table 2, http://links.lww.com/AOSO/A65 and Table 2, respectively. Most studies included patients with various indications for emergency surgery. Surgical procedures ranged from segmental colectomies to subtotal colectomies with or without the creation of a stoma. Reported conversion rates ranged from 0% to 38% across the studies.

TABLE 2.

Indications and Surgical Procedure Characteristics

Study Indication(s) Procedure(s) Laparoscopy, N (%) Open, N (%) Conversion, N (%)
Ballian et al CRC, IBD, CD, other C 341 (9.6) 3211 (90.4)
Cassini et al CD S 36 (60.0) 24 (40.0) 4 (11.1)
Catani et al CRC, IBD, CD, other LHC, RHC, SC, LAR 32 (33.7) 61 (66.3) 2 (6.3)
Cocorullo et al NS C 31 (46.3) 36 (53.7)
Cui et al CRC, NS NS 65 (50.8) 63 (49.2)
Dunker et al IBD STC 10 (31.3) 32 (68.7) 0 (0.0)
Gietelink et al CRC NS 694 (13.4) 4498 (86.6)
Harji et al CRC, IBD, CD, other LHC, RHC, SC, STC, SR, other 33 (51.6) 31 (48.4) 0 (0.0)
Keller et al CRC, CD, IBD, other LHC, RHC, SR 954 (4.2) 21,774 (95.8)
Kim et al CRC LHC, RHC, STC, PC, TC, SR, LAR 11 (22.4) 23 (77.6) 0 (0.0)
Koh et al CRC, CD, other LHC, RHC, SR 23 (50.0) 23 (50.0) 4 (17.4)
Lee et al CD SR 457 (12.2) 3299 (87.8) 175 (38)
Letarte et al CD C, SR 39 (31.2) 86 (68.8) 2 (5.1)
Li et al (2009) CD RHC 6 (3.3) 12 (66.7) 0 (0.0)
Li et al (2015) CRC RHC 10 (28.6) 25 (71.4) 0 (0.0)
Liu et al CRC NS 55 (28.4) 138 (71.5)
Marceau et al IBD STC 40 (45.5) 48 (54.5) 2 (5.0)
Marcello et al IBD TC 19 (39.6) 29 (60.4) 0 (0.0)
Moghadamyeghaneh et al CRC NS 249 (19.3) 1044 (80.7) 80 (28.5)
Nash et al CRC, IBD, CD, other LHC, RHC, TC, STC, SR 36 (52.9) 32 (47.1) 5 (13.9)
Ng et al CRC RHC 14 (32.6) 29 (67.4) 0 (0.0)
Odermatt et al CRC LHC, RHC, STC 36 (33.3) 72 (66.7)
Schloricke et al IP NS 24 (66.7) 12 (33.3)
Stulberg et al CRC, IBD, other LHC, RHC, TC, SR 40 (61.5) 25 (38.5) 4 (9.5)
Sujatha-Bhaskar et al CRC, CD C 1039 (10.4) 8979 (89.6) 44 (4.2)
Turley et al CD S 67 (50.0) 67 (50.0)
Vallance et al CRC NS 3435 (22.1) 12,081 (78.9)
Vennix et al CD SR 39 (33.3) 78 (66.7)
Watanabe et al IBD STC 30 (50.0) 30 (50.0)

Indications and surgical procedure characteristics of the included studies.

C, colectomy; CD, complicated diverticulitis; CRC, colorectal carcinoma; IBD, inflammatory bowel disease; IP, iatrogenic perforation by colonoscopy; LAR, low anterior resection; LHC, left hemicolectomy; NS, not specified acute colonic disease; RHC, right hemicolectomy; SC, segmental colectomy; SR, sigmoid resection; STC, subtotal colectomy; NS, not specified surgical procedure.

Primary Outcome Postoperative Mortality

Twenty-six studies reported the primary outcome of postoperative mortality, of which five studies reported zero events in both the laparoscopic and open surgery group (Supplementary Table 3, http://links.lww.com/AOSO/A65). Therefore, 21 studies were included in the meta-analysis (Fig. 2). The short-term postoperative outcomes, stratified by surgical approach, are presented for each study in Table 2. Laparoscopic surgery resulted in a significantly lower likelihood of postoperative mortality in meta-analysis compared to open surgery, with a pooled OR of 0.44 (95% CI, 0.35–0.54, P < 0.001 and I2 33%, P = 0.07). Sensitivity analyses demonstrated no change in pooled OR for when a fixed effect model was used, if the RCT of Harji et al24 was excluded, and if only “good” quality studies were included.

FIGURE 2.

FIGURE 2.

Postoperative mortality after laparoscopic vs open emergency colorectal surgery. Forest plot of the primary outcome; postoperative mortality. CI, confidence interval.

Secondary Outcomes

The mean LOS with SD was reported in seven studies, whereas the median and IQR or range were reported in 13 studies. When converting the medians and IQR or ranges to estimated means and SDs, meta-analysis showed an SMD of −0.42 (95% CI −0.55 to −0.29, P < 0.001, with I2 84%, P < 0.001) (Fig. 3D-i) in favor of laparoscopy. Due to significant heterogeneity, an additional meta-analysis was performed for studies that reported means and SDs, which showed an SMD of −0.25 between laparoscopic and open surgery (95% CI −0.32 to −0.18, with I2 28%, P = 0.22) (Fig. 3D-ii).

FIGURE 3.

FIGURE 3.

Forest plots of the secondary outcomes after emergency colorectal surgery for a laparoscopic and open approach. (A–D) Forest plots of secondary outcomes after laparoscopic vs open emergency colorectal surgery. (A) Forest plot of overall morbidity. Overall morbidity after emergency colorectal surgery: laparoscopic vs open approach. (B) Forest plot of wound infections. (C) Forest plot of ileus. Ileus after emergency colorectal cancer surgery: laparoscopic vs open approach. (D) Forest plot of length of hospital stay (only studies that reported mean and SD are included in this meta-analysis). (D-i) Length of hospital stay after emergency colorectal surgery: laparoscopic vs open approach. (D-ii) Length of hospital stay after emergency colorectal surgery combining mean and median: laparoscopic vs open approach. CI, confidence interval.

The pooled OR for overall postoperative morbidity showed that there was significantly less morbidity in the laparoscopic group compared to the open surgery group (OR, 0.53; 95% CI, 0.43–0.65, P < 0.001 and I2 68%, P < 0.001) (Fig. 3A). In subgroup analyses for colorectal cancer, complicated diverticulitis and IBD, comparable results favoring laparoscopic surgery were observed (Supplemental Figures 1A–C, http://links.lww.com/AOSO/A63). In meta-analyses for individual patient complications, laparoscopic surgery revealed significantly lower rates for wound infection (OR, 0.63; 95% CI, 0.45–0.88; P = 0.006, and I2 43%, P = 0.04) (Fig. 3B), wound dehiscence (OR, 0.37; 95% CI, 0.18–0.77; P = 0.008 and I2 0%, P = 0.90) (Supplemental Figure 2C, http://links.lww.com/AOSO/A64), postoperative ileus (OR 0.68; 95% CI, 0.51–0.91; P = 0.009 and I2 49%, P = 0.02) (Fig. 3C), pulmonary complications (OR 0.43; 95% CI, 0.24–0.78; P < 0.001; I2 86%, P < 0.001) (Supplemental Figure 2D, http://links.lww.com/AOSO/A64) and cardiac complications (OR, 0.56; 95% CI, 0.35–0.90; P = 0.02 and I2 0%, P = 0.86) (Supplemental Figure 2E, http://links.lww.com/AOSO/A64). There was no significant difference in reinterventions (OR, 0.83; 95% CI, 0.65–1.04; P = 0.11 and I2 22%, P = 0.21) or ICU admissions (OR, 0.49; 95% CI, 0.21–1.14; P = 0.10 and I2 51%, P = 0.09) between the two groups. No meta-analyses were performed for anastomotic leakage, intra-abdominal infection or abscess, and readmission, due to inconsistent definitions, overlapping variables and no expected differences in outcomes.

Long-term Outcomes

There were only six studies that reported long-term outcomes, as presented in Supplementary Table 4, http://links.lww.com/AOSO/A65. Three studies provided long-term oncological outcomes as 3-year overall survival and 3-year recurrence-free survival, without significant difference between the surgical approaches. However, it was decided not to perform meta-analysis in view of the small patient numbers at risk after 3 years of follow-up and differences in calculating survival rates (crude versus cumulative). The other three studies reported incisional hernias, all reporting an advantage for laparoscopy. Again, due to small patient numbers and inconsistency of length of follow-up, no meta-analysis was performed.

DISCUSSION

This systematic review and meta-analysis comprising 28 comparative cohort studies, including several large population-based studies, and 1 feasibility RCT showed that intentional laparoscopic emergency colorectal surgery is associated with decreased postoperative mortality compared to the open approach. Additionally, laparoscopy demonstrated lower rates of overall morbidity, wound infection, wound dehiscence, postoperative ileus, pulmonary and cardiac complications, and reduced LOS compared to open surgery. Comparable results favoring laparoscopic surgery were found in subgroup analyses for colorectal cancer, complicated diverticulitis, and IBD.

Although older retrospective cohort studies comparing laparoscopic and open colorectal surgery in the emergency setting did not find any significant difference in 30-day mortality individually,23,25,26,28,33,34,3642 the pooled OR in this meta-analysis did show reduced postoperative mortality after laparoscopy. This was due to the inclusion of several large population-based studies43,44 and more recently published studies of superior quality according to their NOS scores.27,45,46 Besides, effective training programs, broad implementation, and increasing experience with (advanced) laparoscopy might have contributed to better results when compared to open emergency surgery in more recent studies.

The decreased postoperative mortality following laparoscopic surgery might be a direct consequence of reduced postoperative morbidity. A previous meta-analysis of Pucher et al47 showed that postoperative morbidity after major surgery for various diseases was associated with significantly higher all-cause mortality. Fernandez-Bustamante et al48 showed that pulmonary complications after noncardiothoracic surgery increased the risk of mortality, ICU admission, and prolonged hospital stay. Gietelink et al43 showed that laparoscopic colorectal surgery at a population level decreased the risk of 30-day mortality in both the elective and emergency setting, likely related to a reduction of cardiopulmonary complications. Laparoscopy reduces pain, facilitating deep breathing and coughing, and allows for earlier mobilization, all potentially reducing the risk of pulmonary complications. Similarly, less pain and surgical stress by minimally invasive surgery can reduce cardiac complications, translating into lower postoperative mortality.

Improvements in surgical morbidity after laparoscopy, as observed in the present study are likely explained by less surgical trauma. Not only the size of wounds are limited, but also immune activation is reduced with better preservation of the postoperative immunological defense compared to laparotomy.49,50 This is reflected by the reduced number of postoperative wound infections and wound dehiscence. Also, the observed positive influence on postoperative ileus seems to be directly related to the surgical approach, given the association between the risk of developing small bowel obstruction and the degree of surgical trauma.51,52 Reducing the risk of postoperative ileus is of particular importance in the emergency setting, as some patients will already have a bowel obstruction and others are at risk for paralytic bowel to infectious problems (eg, IBD or diverticulitis). At the moment, there is a lack of studies focusing on long-term adhesion formation after laparoscopy versus open emergency colorectal surgery.

Several studies have demonstrated that laparoscopic colorectal surgery leads to faster patient recovery, and as a consequence to a shorter LOS resulting in lower healthcare costs.53,54 Our pooled analyses also showed that for the emergency setting, the LOS was significantly shorter for laparoscopically treated patients. The reintervention rate was not significantly different between the laparoscopic group and open surgery group. However, the two largest and most highly weighted studies in the meta-analysis individually did show lower reintervention rates in the laparoscopy group.35,44 This illustrates the potential impact on healthcare resources if extending the indications for laparoscopy to the emergency setting.

No meta-analyses on long-term outcomes following emergency colorectal surgery could be performed since these were scarcely reported. One might hypothesize that reduction of (infectious) complications by laparoscopy can translate into less cancer recurrence and improved survival.5559 An inflammatory environment has several similar signaling molecules (such as IL-1 and IL-6) and infection-based immunologic pathways that also play a role in tumor cell invasion, migration, and dissemination.6062 Future studies have to confirm this hypothesis.

Incisional herniation is one of the most common long-term complications after midline laparotomy, with a considerable burden on patients’ QOL and emergency surgery as an important risk factor.6365 Unfortunately, only three studies provided data on this endpoint; however, all three studies showed lower incisional hernia rates following laparoscopy.29,33,41 Retaining the abdominal wall integrity might even be one of the most significant advantages of laparoscopy, in the long run, emphasizing the importance of this endpoint in future studies.

Before concluding that laparoscopic emergency colorectal surgery is superior to the open approach, several limitations need to be addressed. Two factors presumably leading to selection bias in the included studies must be taken into account. First of all, there might be confounding by indication because the initial disease for which the emergency procedure was performed might influence the surgical approach and its outcomes, as patients with a malignant indication likely differ from patients with a benign indication with regard to age and condition. We were not able to assess all outcomes in subgroup analyses based on indication, because outcomes were often not provided separately depending on the underlying disease. Nevertheless, we did perform a subgroup meta-analysis for the combined outcome of overall morbidity. Besides, the preoperative level of illness of the patients might have influenced the type of surgical approach. For example, patients that show signs of sepsis or even a septic shock are probably more likely to undergo open surgery due to the preference of the surgeon or anesthesiologist. Interestingly, the included studies that did report level of illness did not find a significant difference between patients in the laparoscopic and open group for the Mannheim peritonitis score,41 the APACHE score,39 vital clinical parameters as reported by Marcello et al,32 or in the Truelove and Witt’s criteria.36 Studies that did find significant differences in baseline clinical illness severity used propensity score matching to assess the outcomes.26,28,46 Five of these studies were included in the meta-analyses for postoperative mortality, with one of these studies46 showing a significantly lower risk of mortality following laparoscopy (Fig. 2). Another study that adjusted outcomes for clinical illness severity28 showed significantly lower postoperative morbidity after laparoscopy (Fig. 3A).

Second, several of the included studies showed that in the emergency setting, laparoscopic procedures are more often performed in high-volume hospitals with dedicated colorectal emergency services and thus specialized colorectal surgeons.27,6668 Therefore, these results may not represent the outcomes of the general population since emergency cases also present at night. Surgery during night hours can negatively influence postoperative outcomes, probably because of less experienced surgeons preferably performing traditional laparotomy.69 However, we were not able to perform subgroup analysis on timing of surgery, as none of the studies reported outcomes of emergency surgery during night time compared to day time.

One of the main strengths of this systematic review and meta-analysis is that data of over 60,000 patients were pooled, which will be difficult to achieve in a prospective setting. Furthermore, included population-based studies increase the external validity and generalizability of the results. Additionally, a recently published feasibility multicenter, single-blind, parallel-group RCT was included, which showed that emergency laparoscopy is feasible and safe.24 Ideally, larger prospective studies should be conducted; however, it might be difficult to complete such trials with sufficient sample sizes that allow for proper assessment of rare events such as mortality.

In conclusion, this systematic review and meta-analysis showed that emergency laparoscopic surgery for colorectal diseases has a lower postoperative mortality rate, overall morbidity rate, and a shorter length of hospital stay compared to open surgery. However, the almost exclusive use of retrospective observational study designs with inherent biases should be taken into account.

ACKNOWLEDGMENTS

Faridi S. Etten – Jamaludin, the clinical librarian, assisted with assembling the search string. The study protocol was registered in PROSPERO. A.K.W., E.S.Z., E.J.d.G., and P.J.T. wrote the study protocol. A.K.W. and E.S.Z. collected the data. A.K.W., E.S.Z., and E.J.d.G. drafted the article. P.J.T., J.W.T.D., R.A.E.M.T., R.H., and W.A.B. revised the article for important intellectual content. All authors have no conflicts of interest to declare. No funding was received. All authors agreed with the final version of the article.

Supplementary Material

as9-2-e097-s001.pdf (12.7MB, pdf)
as9-2-e097-s002.pdf (22.2MB, pdf)
as9-2-e097-s003.pdf (1.3MB, pdf)

Footnotes

Anne-Loes K. Warps and Emma S. Zwanenburg contributed equally to this article.

Pieter J. Tanis and Elisabeth J. de Groof share senior authorship.

Disclosure: The author declares that they have nothing to disclose.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.annalsofsurgery.com).

REFERENCES

  • 1.Govaert JA, Fiocco M, van Dijk WA, et al. ; Dutch Value Based Healthcare Study Group. Multicenter stratified comparison of hospital costs between laparoscopic and open colorectal cancer resections: influence of tumor location and operative risk. Ann Surg. 2017;266:1021–1028. [DOI] [PubMed] [Google Scholar]
  • 2.Schwenk W, Haase O, Neudecker JJ, et al. Short term benefits for laparoscopic colorectal resection. Cochrane Database Syst Rev. 2005;3:CD003145. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Lacy AM, García-Valdecasas JC, Delgado S, et al. Laparoscopy-assisted colectomy versus open colectomy for treatment of non-metastatic colon cancer: a randomised trial. Lancet. 2002;359:2224–2229. [DOI] [PubMed] [Google Scholar]
  • 4.Nelson H, Sargent DJ, Wieand HS, et al. A comparison of laparoscopically assisted and open colectomy for colon cancer. New Engl J Med. 2004;350:2050–2059. [DOI] [PubMed] [Google Scholar]
  • 5.Abraha I, Binda GA, Montedori A, et al. Laparoscopic versus open resection for sigmoid diverticulitis. Cochrane Database Syst Rev. 2017;11:CD009277. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Sica GS, Iaculli E, Benavoli D, et al. Laparoscopic versus open ileo-colonic resection in Crohn’s disease: short- and long-term results from a prospective longitudinal study. J Gastrointest Surg. 2008;12:1094–1102. [DOI] [PubMed] [Google Scholar]
  • 7.Ingraham AM, Cohen ME, Bilimoria KY, et al. Comparison of hospital performance in nonemergency versus emergency colorectal operations at 142 hospitals. J Am Coll Surg. 2010;210:155–165. [DOI] [PubMed] [Google Scholar]
  • 8.Bakker IS, Snijders HS, Grossmann I, et al. High mortality rates after nonelective colon cancer resection: results of a national audit. Colorectal Dis. 2016;18:612–621. [DOI] [PubMed] [Google Scholar]
  • 9.Xu SB, Jia Z, Zhu YP, et al. Emergent laparoscopic colectomy is an effective alternative to open resection for benign and malignant diseases: a meta-analysis. Indian J Surg. 2017;79:116–123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Chand M, Siddiqui MR, Gupta A, et al. Systematic review of emergent laparoscopic colorectal surgery for benign and malignant disease. World J Gastroenterol. 2014;20:16956–16963. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Moher D, Liberati A, Tetzlaff J, et al. ; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62:1006–1012. [DOI] [PubMed] [Google Scholar]
  • 12.Netherlands Trial Register. Available at: https://www.trialregister.nl/. Accessed January 6, 2021.
  • 13.Clinical Trials, National Institutes of Health, U.S. National Library of Medicine. Available at: https://clinicaltrials.gov/. Accessed January 6, 2021..
  • 14.Wells G, Shea B, O’Connell D, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality if nonrandomized studies in meta-analyses. Available at: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp. Epub ahead of print 2012. doi: 10.2307/632432
  • 15.Viswanathan M, Ansari MT, Berkman ND, et al. Assessing the risk of bias of individual studies in systematic reviews of health care interventions; 2008. Available at: http://www.ncbi.nlm.nih.gov/pubmed/22479713. [PubMed]
  • 16.Penson DF, Krishnaswami S, Jules A, et al. Evaluation and treatment of cryptorchidism comparative effectiveness review number 88. Comparative Effectiveness Review; 2012. Available at: www.effectivehealthcare.ahrq.gov/reports/final.cfm.
  • 17.Mata-Mbemba D, Rohringer T, Ibrahim A, et al. HR-pQCT imaging in children, adolescents and young adults: systematic review and subgroup meta-analysis of normative data. PLoS One. 2019;14:e0225663. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Vivekanantham A, Edwin C, Pincus T, et al. The association between headache and low back pain: a systematic review. J Headache Pain. 2019;20:82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Cochrane handbook for systematic reviews of Interventions. Part 2: General methods for Cochrane reviews. Chapter 9: Analysing data and undertaking meta-analyses. Available at: https://handbook-5-1.cochrane.org/chapter_9/9_analysing_data_and_undertaking_meta_analyses.htm. Assessed January 6, 2021.
  • 20.Page MJ, Higgins JPT. SJ. Chapter 13: Assessing risk of bias due to missing results in a synthesis. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ WV. eds. Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane, 2021. Available at: www.training.cochrane.org/handbook. [Google Scholar]
  • 21.Wan X, Wang W, Liu J, et al. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol. 2014;14:135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Deeks JJ, Higgins JPT. AD (editors). Chapter 10: Analysing data and undertaking meta-analyses. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ WV, eds. Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane, 2021. doi: 10.1002/9781119536604 [Google Scholar]
  • 23.Stulberg JJ, Champagne BJ, Fan Z, et al. Emergency laparoscopic colectomy: does it measure up to open? Am J Surg. 2009;197:296–301. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Harji DP, Marshall H, Gordon K, et al. ; LaCeS Collaborators. Laparoscopic versus open colorectal surgery in the acute setting (LaCeS trial): a multicentre randomized feasibility trial. Br J Surg. 2020;107:1595–1604. [DOI] [PubMed] [Google Scholar]
  • 25.Liu Z, Kang L, Huang M, et al. No advantages of laparoscopy for left-sided malignant colonic obstruction compared with open colorectal resection in both short-term and long-term outcomes. Med Oncol. 2014;31:213. [DOI] [PubMed] [Google Scholar]
  • 26.Turley RS, Barbas AS, Lidsky ME, et al. Laparoscopic versus open Hartmann procedure for the emergency treatment of diverticulitis: a propensity-matched analysis. Dis Colon Rectum. 2013;56:72–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Vallance AE, Keller DS, Hill J, et al. Role of emergency laparoscopic colectomy for colorectal cancer: a population-based study in England. Ann Surg. 2019;270:172–179. [DOI] [PubMed] [Google Scholar]
  • 28.Vennix S, Lips DJ, Di Saverio S, et al. Acute laparoscopic and open sigmoidectomy for perforated diverticulitis: a propensity score-matched cohort. Surg Endosc. 2016;30:3889–3896. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Dunker MS, Bemelman WA, Slors JF, et al. Laparoscopic-assisted vs open colectomy for severe acute colitis in patients with inflammatory bowel disease (IBD): a retrospective study in 42 patients. Surg Endosc. 2000;14:911–914. [DOI] [PubMed] [Google Scholar]
  • 30.Kim IY, Kim BR, Kim YW. Outcomes of laparoscopic and open surgery for colorectal cancer in the emergency setting. In Vivo. 2015;29:295–300. [PubMed] [Google Scholar]
  • 31.Marceau C, Alves A, Ouaissi M, et al. Laparoscopic subtotal colectomy for acute or severe colitis complicating inflammatory bowel disease: a case-matched study in 88 patients. Surgery. 2007;141:640–644. [DOI] [PubMed] [Google Scholar]
  • 32.Marcello PW, Milsom JW, Wong SK, et al. Laparoscopic total colectomy for acute colitis: a case-control study. Dis Colon Rectum. 2001;44:1441–1445. [DOI] [PubMed] [Google Scholar]
  • 33.Nash GM, Bleier J, Milsom JW, et al. Minimally invasive surgery is safe and effective for urgent and emergent colectomy. Colorectal Dis. 2010;12:480–484. [DOI] [PubMed] [Google Scholar]
  • 34.Ng SS, Lee JF, Yiu RY, et al. Emergency laparoscopic-assisted versus open right hemicolectomy for obstructing right-sided colonic carcinoma: a comparative study of short-term clinical outcomes. World J Surg. 2008;32:454–458. [DOI] [PubMed] [Google Scholar]
  • 35.Sujatha-Bhaskar S, Alizadeh RF, Koh C, et al. The growing utilization of laparoscopy in emergent colonic disease. Am Surg. 2017;83:1068–1073. [PubMed] [Google Scholar]
  • 36.Watanabe K, Funayama Y, Fukushima K, et al. Hand-assisted laparoscopic vs. open subtotal colectomy for severe ulcerative colitis. Dis Colon Rectum. 2009;52:640–645. [DOI] [PubMed] [Google Scholar]
  • 37.Schlöricke E, Bader FG, Hoffmann M, et al. Offen chirurgische versus laparoskopische Versorgung der iatrogenen Kolonperforation – Ergebnisse nach 13 Jahren Erfahrungen. Zentralblatt für Chirurgie. 2013;138:257–261. [DOI] [PubMed] [Google Scholar]
  • 38.Odermatt M, Miskovic D, Siddiqi N, et al. Short- and long-term outcomes after laparoscopic versus open emergency resection for colon cancer: an observational propensity score-matched study. World J Surg. 2013;37:2458–2467. [DOI] [PubMed] [Google Scholar]
  • 39.Letarte F, Hallet J, Drolet S, et al. Laparoscopic versus open colonic resection for complicated diverticular disease in the emergency setting: a safe choice? A retrospective comparative cohort study. Am J Surg. 2015;209:992–998. [DOI] [PubMed] [Google Scholar]
  • 40.Cocorullo G, Falco N, Tutino R, et al. Open versus laparoscopic approach in the treatment of abdominal emergencies in elderly population. G Chir. 2016;37:108–112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Cassini D, Miccini M, Manoochehri F, et al. Emergency Hartmann’s procedure and its reversal: a totally laparoscopic 2-step surgery for the treatment of Hinchey III and IV Diverticulitis. Surg Innov. 2017;24:557–565. [DOI] [PubMed] [Google Scholar]
  • 42.Catani M, De Milito R, Romagnoli F, et al. Laparoscopic colorectal surgery in urgent and emergent settings. Surg Laparosc Endosc Percutan Tech. 2011;21:340–343. [DOI] [PubMed] [Google Scholar]
  • 43.Gietelink L, Wouters MW, Bemelman WA, et al. ; Dutch Surgical Colorectal Cancer Audit Group. Reduced 30-day mortality after laparoscopic colorectal cancer surgery: a population based study from the Dutch Surgical Colorectal Audit (DSCA). Ann Surg. 2016;264:135–140. [DOI] [PubMed] [Google Scholar]
  • 44.Ballian N, Weisensel N, Rajamanickam V, et al. Comparable postoperative morbidity and mortality after laparoscopic and open emergent restorative colectomy: outcomes from the ACS NSQIP. World J Surg. 2012;36:2488–2496. [DOI] [PubMed] [Google Scholar]
  • 45.Moghadamyeghaneh Z, Talus H, Ballantyne G, et al. Short-term outcomes of laparoscopic approach to colonic obstruction for colon cancer. Surg Endosc. 2021;35:2986–2996. [DOI] [PubMed] [Google Scholar]
  • 46.Lee YF, Brown RF, Battaglia M, et al. Laparoscopic versus open emergent sigmoid resection for perforated diverticulitis. J Gastrointest Surg. 2020;24:1173–1182. [DOI] [PubMed] [Google Scholar]
  • 47.Pucher PH, Aggarwal R, Qurashi M, et al. Meta-analysis of the effect of postoperative in-hospital morbidity on long-term patient survival. Br J Surg. 2014;101:1499–1508. [DOI] [PubMed] [Google Scholar]
  • 48.Fernandez-Bustamante A, Frendl G, Sprung J, et al. Postoperative pulmonary complications, early mortality, and hospital stay following noncardiothoracic surgery: a multicenter study by the Perioperative Research Network Investigators. JAMA Surg. 2017;152:157–166. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Novitsky YW, Litwin DE, Callery MP. The net immunologic advantage of laparoscopic surgery. Surg Endosc. 2004;18:1411–1419. [DOI] [PubMed] [Google Scholar]
  • 50.Wang G, Jiang Z, Zhao K, et al. Immunologic response after laparoscopic colon cancer operation within an enhanced recovery program. J Gastrointest Surg. 2012;16:1379–1388. [DOI] [PubMed] [Google Scholar]
  • 51.Angenete E, Jacobsson A, Gellerstedt M, et al. Effect of laparoscopy on the risk of small-bowel obstruction a population-based register study . Arch Surg. 2012;147:359–365. [DOI] [PubMed] [Google Scholar]
  • 52.Gutt CN, Oniu T, Schemmer P, et al. Fewer adhesions induced by laparoscopic surgery? Surg Endosc. 2004;18:898–906. [DOI] [PubMed] [Google Scholar]
  • 53.Dobson MW, Geisler D, Fazio V, et al. Minimally invasive surgical wound infections: laparoscopic surgery decreases morbidity of surgical site infections and decreases the cost of wound care. Colorectal Dis. 2011;13:811–815. [DOI] [PubMed] [Google Scholar]
  • 54.Keller DS, Delaney CP, Hashemi L, et al. A national evaluation of clinical and economic outcomes in open versus laparoscopic colorectal surgery. Surg Endosc. 2016;30:4220–4228. [DOI] [PubMed] [Google Scholar]
  • 55.Law WL, Choi HK, Lee YM, et al. The impact of postoperative complications on long-term outcomes following curative resection for colorectal cancer. Ann Surg Oncol. 2007;14:2559–2566. [DOI] [PubMed] [Google Scholar]
  • 56.Artinyan A, Orcutt ST, Anaya DA, et al. Infectious postoperative complications decrease long-term survival in patients undergoing curative surgery for colorectal cancer: a study of 12,075 patients. Ann Surg. 2015;261:497–505. [DOI] [PubMed] [Google Scholar]
  • 57.Klaver CE, Gietelink L, Bemelman WA, et al. ; Dutch Surgical Colorectal Audit Group. Locally advanced colon cancer: evaluation of current clinical practice and treatment outcomes at the population level. J Natl Compr Canc Netw. 2017;15:181–190. [DOI] [PubMed] [Google Scholar]
  • 58.Mirnezami A, Mirnezami R, Chandrakumaran K, et al. Increased local recurrence and reduced survival from colorectal cancer following anastomotic leak: systematic review and meta-analysis. Ann Surg. 2011;253:890–899. [DOI] [PubMed] [Google Scholar]
  • 59.Klaver CEL, Wasmann KATGM, Verstegen M, et al. Postoperative abdominal infections after resection of T4 colon cancer increase the risk of intra-abdominal recurrence. Eur J Surg Oncol. 2018;44:1880–1888. [DOI] [PubMed] [Google Scholar]
  • 60.Wu Y, Zhou BP. Inflammation: a driving force speeds cancer metastasis. Cell Cycle. 2009;8:3267–3273. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Coussens LM, Werb Z. Inflammation and cancer. Nature. 2002;420:860–867. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Sista F, Schietroma M, Santis GD, et al. Systemic inflammation and immune response after laparotomy vs laparoscopy in patients with acute cholecystitis, complicated by peritonitis. World J Gastrointest Surg. 2013;5:73–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Fink C, Baumann P, Wente MN, et al. Incisional hernia rate 3 years after midline laparotomy. Br J Surg. 2014;101:51–54. [DOI] [PubMed] [Google Scholar]
  • 64.Mingoli A, Puggioni A, Sgarzini G, et al. Incidence of incisional hernia following emergency abdominal surgery. Ital J Gastroenterol Hepatol. 1999;31:449–453. [PubMed] [Google Scholar]
  • 65.Darnis B, Mohkam K, Golse N, et al. Long-term abdominal wall benefits of the laparoscopic approach in liver left lateral sectionectomy: a multicenter comparative study. Surg Endosc. 2021;35:5034–5042. [DOI] [PubMed] [Google Scholar]
  • 66.Osagiede O, Haehn DA, Spaulding AC, et al. Influence of surgeon specialty and volume on the utilization of minimally invasive surgery and outcomes for colorectal cancer: a retrospective review. Surg Endosc. Epub ahead of print 2020. doi: 10.1007/s00464-020-08039-9 [DOI] [PubMed] [Google Scholar]
  • 67.Archampong D, Borowski D, Wille-Jørgensen P, et al. Workload and surgeon´s specialty for outcome after colorectal cancer surgery. Cochrane Database Syst Rev. 2012:Cd005391. [DOI] [PubMed] [Google Scholar]
  • 68.Keller DS, Pedraza R, Flores-Gonzalez JR, et al. The current status of emergent laparoscopic colectomy: a population-based study of clinical and financial outcomes. Surg Endosc. 2016;30:3321–3326. [DOI] [PubMed] [Google Scholar]
  • 69.Cortegiani A, Ippolito M, Misseri G, et al. Association between night/after-hours surgery and mortality: a systematic review and meta-analysis. Br J Anaesth. 2020;124:623–637. [DOI] [PubMed] [Google Scholar]

Articles from Annals of Surgery Open are provided here courtesy of Wolters Kluwer Health

RESOURCES