Interventions to reduce the incidence of surgical site infection in colorectal resections: systematic review with multicomponent network meta-analysis (INTRISSI): study protocol

Abstract

Objective

To assess the relative contribution of intravenous antibiotic prophylaxis, mechanical bowel preparation, oral antibiotic prophylaxis, and combinations thereof towards the reduction of surgical site infection (SSI) incidence in elective colorectal resections.

Methods and analysis

A systematic search of randomised controlled trials comparing interventions to reduce SSI incidence will be conducted with predefined search terms in the following databases: MEDLINE, LILACS, Cochrane Central Register of Controlled Trials (CENTRAL) and the Cochrane Database of Systematic Reviews (CDSR). Additionally, several online databases will be searched for ongoing trials, and conference proceedings and reference lists of retrieved articles will be hand searched. The title–abstract screening will be partly performed by means of a semiautomated supervised machine learning approach, which will be trained on a subset of the identified titles and abstracts identified through traditional screening methods.

The primary analysis will be a multicomponent network meta-analysis, as we expect to identify studies that investigate combinations of interventions (eg, mechanical bowel preparation combined with oral antibiotics) as well as studies that focus on individual components (mechanical bowel preparation or oral antibiotics). By means of a multicomponent network meta-analysis, we aim at estimating the effects of the separate components along the effects of the observed combinations. To account for between-trial heterogeneity, a random-effect approach will be combined with inverse variance weighting for estimation of the treatment effects. Associated 95% CIs will be calculated as well as the ranking for each component in the network using P scores. Results will be visualised by network graphics and forest plots of the overall pairwise effect estimates. Comparison-adjusted funnel plots will be used to assess publication bias.

Ethics and dissemination

Ethical approval by the Ethical Committee of the Medical Faculty of the Martin-Luther-University Halle-Wittenberg (ID of approval: 2021–148). Results shall be disseminated directly to decision-makers (eg, surgeons, gastroenterologists, wound care specialists) by means of publication in peer-reviewed journals, presentation at conferences and through the media (eg, radio, TV, etc).

PROSPERO registration number

CRD42021267322.

Strengths and limitations of this study.

• Literature screening is supported by machine learning, which is a new and highly innovative technique saving work and time.

• The multicomponent network meta-analysis integrates and compares all available evidence on how effective the different interventions are in preventing surgical site infections.

• Results will be rated and discussed with patient representatives.

• No individual patient data will be available from trials.

Introduction

Colorectal resections are some of the most frequently performed operations in abdominal surgery. For 2018, in Germany, the annual number of colorectal resections for all causes reached 350 803.¹ Due to the microbiome inherent to the colon and rectum, postoperative surgical site infections (SSIs) are a frequent problem. According to the commonly used definition of the Centres for Disease Control (CDC), they comprise infections of the incision, fascia and muscle layer, or the organ space.² It is estimated that SSIs occur in up to 40% of colorectal resections,³ amounting to approximately 130 000 annual cases of SSI in Germany alone.⁴

SSIs have a relevant impact on both patients and healthcare systems. Depending on severity, they require additional interventions, prolong the hospital stay, increase treatment and societal costs, negatively impact quality of life, lead to temporary or permanent disability or can even be life threatening. The median prolongation of hospital stays due to SSIs after colorectal surgery is estimated to be 7 days,⁵ and the quality-adjusted life years lost for patients suffering an SSI 0.93.⁶ The cost of an SSI is estimated at around US$30 000 in the USA⁶ and between €926 and €65 114 in Germany.⁷

Several interventions have been used with the aim of reducing the SSI risk in elective colorectal resections. However, the evidence is conflicting and to some aspects contradictory. Intravenous antibiotics prior to skin incision are considered indispensable based on high-level evidence from studies comparing it to no intravenous antibiotic administration.³ Mechanical bowel preparation (MBP) using a drinkable cleansing solution has been assessed in a recent meta-analysis comprising both randomised controlled trials (RCTs) and observational studies. It failed to show a lower SSI incidence in patients receiving MBP.⁸ There is evidence that oral antibiotic prophylaxis (OAP) in combination with MBP prior to resection reduces SSI incidence to a larger extent than MBP alone.⁹ This has led to the recommendation of the combination in the WHO guidelines for SSI prevention.¹⁰ Yet, when compared with intravenous antibiotic prophylaxis, OAP is associated with higher SSI incidence.¹¹ A recent meta-analysis aimed at assessing the impact of OAP with or without MBP compared with different other prophylactic strategies on the incidence of SSI and other postoperative complications.¹² The authors concluded that OAP is associated with lower SSI incidence. However, the interpretation of the results is limited because no network meta-analysis was done. This methodology was applied by Toh et al for a comparison of different prophylactic interventions.¹³ The analysis found that MBP with OAP was associated with the lowest SSI risk compared with OAB alone, MBP alone or no preparation. However, the effect of intravenous antibiotics was not assessed and the methods used did not allow calculating the relative contribution of the single interventions to the observed effects.

In summary, there is substantial evidence comparing different interventions and combinations thereof for SSI prevention in elective colorectal surgery. Yet, no comprehensive analysis of this evidence using appropriate methods for discerning the true effects of the single interventions or combinations has been done so far. Traditional network meta-analysis approaches either lump such combinations into classes of treatments with high variation between studies contributing information, or treat all combinations as separate nodes by splitting the network. Recently, an approach developed for network meta-analysis of multicomponent interventions¹⁴ has been formalised.¹⁵ This multicomponent network meta-analysis (CNMA) estimates the separate components of which treatments consist, for example, MBP, intravenous antibiotics and OAP, along with treatment combinations actually used in identified RCTs. CNMA is, therefore, the only approach that allows estimating treatment effects of a given component relative to a reference component, of combinations of components compared with a reference component and of all possible treatment contrasts based on the estimation results and the network structure. It is the only method that can validly answer the research question regarding SSI prevention in elective colorectal resection.

Another important aspect targeted by this study is the burdensome and time-consuming title–abstract screening in systematic reviews. Although machine learning (ML) has developed rapidly in recent years and has been proposed for usage in systematic reviews,^16–18 there is only a limited number of studies actually applying text mining in combination with supervised learning in medical research.^{17 19} The project evaluates the practicability of applying natural language processing procedures and ML techniques to abstract screening in the scope of a real-world example. We aim at providing a standardised workflow to support abstract screening with advanced ML techniques.

Methods and analysis

Search strategies and information sources

A computer-based literature search will be performed in several databases, including the Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Database of Systematic Reviews (CDSR) from The Cochrane Library, MEDLINE (1966 to present), LILACS (Literatura Latinoamericana y del Caribe en Ciencias de la Salud), Current Contents/Clinical Medicine (1990 to present) and Web of Science (1945 to present). The search will be limited to studies in humans. No language restrictions will apply. The Cochrane Highly Sensitive Search Strategy for identifying randomised trials in MEDLINE, Sensitivity maximising version, NCBI Platform, will be employed with predefined search terms (online supplemental file). It will be adapted for the other databases searched. Moreover, the following online databases of ongoing trials will be searched: www.clinicaltrials.nci.nih.gov; www.centerwatch.com; www.trialscentral.org; www.controlledtrials.com; www.eortc.be; www.studien.de and wwwgermanctrde. Reference lists of retrieved articles will be scanned for further eligible trials (backward search) and citations of identified trials will be checked for inclusion (forward search). Experts in the field will be contacted about any unpublished or ongoing studies

Study selection

This CNMA is limited to RCTs, which are the only study design able to provide unbiased evidence for the research question. Due to the nature of the interventions and comparator under study, blinding of either the patient or the treating physician is not possible for all interventions and is, therefore, not considered an inclusion or exclusion criterion. There are no restrictions regarding minimal follow-up time or study size.

Study selection will be partly performed using ML methods in a semiautomated screening procedure due to the large number of citations expected from our search strategy. First, the identified studies will be randomised into three data sets: training data set, validation data set and screening data set, with equal sizes of 33.3%. In the training set, two independent reviewers will assess title, keywords and abstracts of all retrieved studies and decide which studies are included into the systematic review. Any disagreements will be resolved by consensus or consultation with a third reviewer.

For the purpose of computer-based text classification based on the abstracts, the texts will be preprocessed to achieve consistent format by means of various cleaning approaches, such as transformation to lower case and removing punctuation, symbols, numbers and stop words. Text reduction will be performed by using stemming techniques and a document-term matrix will be created as input for the ML methods. Three ML methods will be applied independently to predict whether a particular study is included into the systematic review or not. Those methods are regularised logistic regression,²⁰ kernel-based support vector machines²¹ and tree-based random forests.²² These three approaches stem from different sectors of ML and, therefore, a wide range of methods is covered. If appropriate, the number of applied algorithms can be extended, for example, if no sufficiently high performance can be achieved by at least one of these algorithms. After tuning the algorithms using repeated cross-validation, the ML techniques in the validation set will be assessed by using the area under the curve (AUC) as performance measure. To consider the imbalanced class distribution, the Precision-Recall (PR) curve²³ will be applied additional to the receiver operating characteristics (ROC) curve to computing the AUC. The corresponding PR and ROC curve will be plotted and key numbers such as the AUC, sensitivity, specificity, accuracy and the worked saved over sampling²⁴ will be reported to evaluate the algorithms’ quality. Per algorithm, the 10% of citations with the largest difference between human decision (0 or 1 for exclude or include) and the predicted (continuous) inclusion probability will be identified and the validity of human-based and algorithm-based decisions will be re-evaluated. This procedure will be repeated by adding a further 10% of unseen, randomly selected citations to the training set. If the performance of at least one ML algorithm is deemed acceptable, the citations in the screening set need to be screened by only one human, and the second reviewer is replaced by the best-performing ML method. For all citations identified as eligible in title–abstract screening, full-text will be retrieved and scrutinised by two independent reviewers. Any disagreements will be resolved by consensus or by consultation with a third reviewer. The entire process of study retrieval, inclusion and exclusion will be displayed in a flowchart as stipulated by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Network Meta-Analysis statement.²⁵

Population

To be included, trials need to be conducted on patients undergoing elective, that is, non-emergent, planned, colorectal resection. Resection will be defined as removal of at least a segment of the colon or rectum, with or without primary anastomosis and with or without protective ostomy placement. There will be no limitations regarding the underlying disease constituting the indication for resection. Thus, both malignant diseases such as colon cancer and benign diseases such as diverticulitis will be included.

Intervention(s)

1. intravenous antibiotic prophylaxis

2. MBP

3. OAP

   or any combination of (1), (2) and (3).

Comparator(s)

No intervention, defined as the absence of any of (1), (2) or (3).

To be included in this CNMA, trials must either compare any of the interventions or combinations thereof directly with another or with no intervention or combinations thereof; or one of the trial arms must be the comparator. A network like the one illustrated in figure 1A is expected to be identified in the systematic literature review. In that network, the treatment nodes are defined mostly by combinations of separate treatment components, and the estimation results need to be interpreted as treatment interaction effects. Besides these interaction effects, the treatment effects for the separate components by means of CNMA will also be estimated, which is visualised in figure 1B.

Figure 1.

(A) Network of treatments expected to be identified through the systematic review. (B) Network of separate components that will be estimated through the component network meta-analysis.

Outcomes

The primary outcome will be SSI of any severity.

Secondary outcomes will be:

• Severity of SSI according to the CDC classification (superficial, deep incisional, organ space).²

• Anastomotic failure.

• Ileus.

• Clostridium difficile infection.

• Postoperative mortality.

• Postoperative morbidity (any in-hospital complication classified as Clavien-Dindo grade I-IV²⁶ or with a comparable classification).

• Reoperation.

• Hospital readmission.

• Hospital length of stay.

• Postoperative length of stay.

• Quality of life (as measured in the single studies).

SSI is the prespecified primary outcome because a direct effect of the tested interventions is assumed.

As detailed in the section ‘Patient involvement’, patient representatives will rank subjective importance of the available secondary outcomes. These rankings will be used to identify highly patient-relevant outcomes on which we will place special emphasis in the interpretation and discussion of results. Regardless of this ranking, the different outcomes reflect different aspects of the clinical course and potential complications following colorectal resection. SSI is commonly classified into three severity grades.² The tested interventions might have different effects on SSI of different severity. Anastomotic failure is a dreaded complication in colorectal surgery, which leads to organ space SSI and is assumed equally amenable to measures reducing SSI incidence. Ileus is a common postoperative problem after colorectal resection and might be triggered by alterations in the colorectal microbiome or mechanical irritations, which are assumed to take place in consequence of SSI prevention measures. Postoperative mortality and morbidity are highly relevant when evaluating colorectal resections. For their assessment, the Clavien-Dindo scheme, a validated and widely used classification of perioperative complications, will preferably be used.²⁶ Reoperation, hospital readmission and both overall and postoperative length of stay are all directly patient-relevant parameters of quality of care. Quality of life is an important outcome, as it is a direct reflection of a patient’s well-being.

Quality assessment and data extraction

Two independent reviewers will assess study quality/risk of bias following Cochrane recommendations.²⁷ Five specific domains of bias will be investigated with the Cochrane risk of bias tool Version 2. Based on this assessment, each reviewer assigns an overall level of risk of bias to each study with respect to the primary outcome. This overall risk of bias is defined as the least favourable assessment across five domains of bias, with each domain being assigned low risk of bias, some concerns or high risk of bias. Bias level will be used as a quality measurement for each study in sensitivity and subgroup analyses. Published aggregate data will be extracted from full texts of publications. Two reviewers will extract data independently by using a standardised extraction form and will consult a third reviewer if arbitration is required to reach consensus. The form will compile the following items, if available, separately for each study arm:

• General information on the study: title, authors, contact address, funding sources, language, publication status, year of publication, place(s) and year(s) of study conduction.

• Study design issues: inclusion/exclusion criteria, randomisation, risk of bias, length of study/follow-up period.

• Baseline characteristics of participants: size of intervention and comparison group, and for each group, the distribution of age, sex, WHO performance status or American Society of Anesthesiologists classification, underlying disease; in case of malignant disease: histology, tumour location (right-sided colon, left-sided colon, rectum), Tumor Node Metastasis (TNM) and Union for International Cancer Control (UICC) stage, neoadjuvant therapy; details of the performed resection (extent, surgical access (open/minimally invasive), construction of anastomosis (yes/no), protective ostomy placement (yes/no)).

• Characteristics of the intervention: details of intravenous antibiotic administration, MBP, OAP including the administered compound and its dosage.

• Loss to follow-up.

• Incidence and precision estimate of SSI of any severity.

• Incidence and precision estimate of the three SSI severity grades according to the CDC classification.²

• Incidence and precision estimates of anastomotic failure, ileus and clostridium difficile infection.

• Postoperative mortality.

• Postoperative morbidity (any in-hospital complication classified as Clavien-Dindo grade I-IV²⁶ or with a comparable classification).

• Incidence and precision estimates of reoperation and hospital readmission

• Hospital length of stay (absolute number of days and precision estimate).

• Postoperative length of stay (absolute number of days and precision estimate).

• Quality of life, as measured within the single trials.

The data extraction form will be pilot tested on two retrieved studies and, if needed, be revised.

Multicomponent network meta-analysis

As described in figure 1, the network will presumably include nodes that consist of combinations of several treatment components (panel A), while the aim is to estimate treatment effects related to the basic components in addition to interaction effects. Assuming an additive relation between combinations of basic components, a random-effects CNMA as described by Welton et al¹⁴ and Rücker et al,²⁸ using the frequentist implementation¹⁵ incorporated in the R²⁹ extension netmeta,³⁰ will be performed. By using this model, multiarm trials can be incorporated and mixed effects for basic and combined components are estimated.

The effect size with respect to dichotomous and categorical outcomes (such as the primary outcome SSI incidence) will be measured with ORs with 95% CIs. We will extract ORs whenever they are reported in the identified trials preferably from adjusted models. Otherwise, for example, in cases where different effect measures such as the risk ratio are reported, they will be calculated using extracted frequencies and sample sizes in the trial arms. For continuous outcomes (eg, length of hospital stay), the standardised mean difference with its 95% CI will be calculated. Ordinal endpoints (eg, quality of life scores) will be treated either as dichotomous events or as continuous data, depending on the number of categories observed as well as the numbers falling into each category. The network meta-analysis model will include random effects to account for possible variation between trials due to clinical or statistical heterogeneity. Basic components, as well as the combinations addressed in primary trials, will be estimated. However, edges in the network that are informed by direct evidence will be compared with results from pairwise meta-analysis using the method of Bucher to assess potential inconsistency in the network. The treatment options will be ranked using the P score. Publication bias will be explored by evaluating funnel plot asymmetry if a sufficient number of studies is available.

Subgroup and sensitivity analyses

With respect to the network meta-analyses and the primary outcome SSI incidence, subgroup analyses stratified for the study-level covariate type of resection (open vs minimally invasive, colon vs rectum) and underlying disease (malignant vs benign) as well as for different characteristics of the single interventions (high vs low-volume solutions for MBP, different classes of antibiotics for intravenous and oral administration) will be conducted. Other subgroup analyses will be defined based on exploratory analyses of the available data. For all outcomes, sensitivity analyses based on the risk of bias assigned to studies as described above (low, some concerns, high) will be performed. All statistical analyses will be conducted with R Version 4.1.1 or higher²⁹ and its extensions netmeta,³⁰ caret³¹ and tidyverse,³² and potentially other required extensions.

A ‘summary of findings’ table will be produced according to the methodology stipulated in the Cochrane Handbook for Systematic Reviews of Interventions.³³ It will provide information on the quality of evidence using the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) system, on the effect magnitude of interventions and on what data are available with regards to the primary and relevant secondary outcomes, for both basic and combined components.

Ethics and dissemination

Ethics approval has been obtained from the Ethical Committee, Medical Faculty, Martin-Luther-University Halle-Wittenberg. The study is registered with PROSPERO.

Strategies for data sharing and dissemination of results

Aggregate data from single trials will be combined in a dedicated database, will be stored in a repository and will on request be made available to other researchers for secondary analyses. Results shall be disseminated directly to decision-makers such as surgeons, gastroenterologists, wound care specialists, etc by means of publication in peer-reviewed journals. The means of dissemination will be presentations at national and international conferences as well as specific events. In particular, a virtual or on-site symposium where the results of the analysis will be presented and discussed among decision-makers is planned. Results will be actively presented to the bodies in charge of national and international treatment guidelines. Because results are expected to have a direct and relevant impact on patients’ decision-making, we will specifically communicate them to patients and the public. Possible media of dissemination are health-specific sections of newspapers, radio and TV programmes as well as a direct approach through patients’ organisations.

Patient involvement

While SSI is the defined primary outcome of this study, several secondary outcomes will be assessed as well. Patient involvement is crucial in order to define the relevance of outcomes to patients. A staged approach regarding patient involvement will be employed. During the literature review, all pre-specified outcomes will be considered. After all data are extracted, available outcomes including how they were collected (eg, specific quality of life indices) will be listed. This list will be a basis for a discussion with patient representatives recruited through the patient organisation Deutsche ILCO e.V. In particular, a focus group discussion with at least five patient representatives will be conducted. This discussion will serve to rank the subjective importance of available outcomes to patients. A ranking scale will be devised by the assignment of points to each outcome by the single participants. Regarding outcomes that can be measured in multiple ways, such as quality of life, the specific measurement available from the trials will also be discussed and judged by the patient representatives. After completion of the analyses, results will be discussed again in the framework of a focus group discussion with patient representatives from Deutsche ILCO e.V. Similar to the first discussion, the importance of the results of the single outcomes will be ranked by assigning points in the light of the specific result. Both rankings will be reported in all presentations of results. It is planned to present results not only to a scientific audience but also to patients and their next of kin through appropriate media and in dedicated settings like information events.

Ethics statements

Patient consent for publication

Not applicable.