Quality of evidence supporting Surviving Sepsis Campaign Recommendations

Abstract

Introduction

The Surviving Sepsis Campaign (SSC) guidelines, released in 2017, are a combination of expert opinion and evidence-based medicine, adopted by many institutions as a standard of practice. The aim was to analyse the quality of evidence supporting recommendations on the management of sepsis.

Methods

The strength and quality of evidence (high, moderate, low-very low and best practice statements) of each recommendation were extracted. Randomised controlled trials were required to qualify as high-quality evidence.

Results

A total of 96 recommendations were formulated, and 87 were included. Among thirty-one (43%) strong recommendations, only 15.2% were supported by high-quality evidence. Overall, thirty-seven (42.5%) recommendations were based on low-quality evidence, followed by 28 (32.2%) based on moderate-quality, 15 (17.2%) were best practice statements and only seven (8.0%) were supported by high-quality evidence. Randomised controlled trials supported 21.4%, 9.5% and 8.6% recommendations on mechanical ventilation, resuscitation, and management/adjuvant therapy, respectively. In contrast, none high-quality evidence recommendation supported antimicrobial/source control (82.4% were low-very low evidence or best practice statements), and nutrition.

Conclusions

In the SSC guidelines most recommendations were informed by indirect evidence and non-systematic observations. While awaiting trials results, Delphi-like approaches or multi-criteria decision analyses should guide recommendations.

1. Introduction

Sepsis is a multi-factorial, life-threatening syndrome that arises from the body's response to infection, leading to organ dysfunction [1]. Sepsis and septic shock, the severe forms of sepsis, are medical emergencies. The World Health Organization (WHO) estimates 30.7 million sepsis cases annually, with at least six million deaths [2]. In 2016, the WHO classified sepsis as a global health priority and have urged for the implementation of measures to improve prevention, diagnosis and management of sepsis [2].

Multiple clinical practice guidelines (CPG) have been developed to guide clinician care and management for patients with sepsis and septic shock. The first set of guidelines published by the Surviving Sepsis Campaign (SCC) were published in 2004 and revised in 2008 and 2012. The current guidelines [3] were published in 2017 and are based on updated research evidence. The quality of the evidence supporting each recommendation was assessed using the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) system [4], which takes into consideration the risk of bias, inconsistency, indirectness, imprecision and publication bias. This contributed to the formulation of either a strong or weak recommendation after weighing up the risks and benefits, patient preferences, cost, feasibility and practicality of the intervention. Recent research has found a majority of recommendations in CPGs in other medical fields, such as hospital-acquired pneumonia and ventilator-associated pneumonia [5], to be supported by low-quality evidence, with only a small number based on high-quality randomised controlled trials (RCT). As CPGs are used to guide patient care and management, it is imperative that these recommendations are based on high-quality evidence, namely well-conducted RCTs.

The main objective of this study is to evaluate the quality of evidence supporting the recommendations from the 2016 Surviving Sepsis Campaign and inform the relevant stakeholders of areas requiring further research in order to provide stronger evidence for the future updates. This is part of a broader research project of quality of evidence evaluation from different CPGs in respiratory and critical care medicine [5], [6], [7]. Recognising guidelines as an important tool to complement clinical reasoning and improve patient care, the aim of this project is to call attention to the weaknesses and to contribute to their future refinement.

2. Methods

The SSC guidelines [3] were identified and downloaded from PubMed. The recommendations for each guideline were abstracted by a single reviewer (EX) and supervised by one of the authors (ST). Questions formulated without reporting specific recommendations, or regarding screening, diagnosis and goals of care were excluded. A flow chart selection following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses [8] (PRISMA) guidelines was conducted.

The reviewers then recorded the level of evidence (LOE) [9] and strength of each recommendation. The LOE was clearly classified as high, moderate, low-very low quality, as determined by the guideline development team, so guideline content was not required to be reviewed. This was translated into LOE A (high; RCTs), LOE B (moderate; downgraded RCTs or upgraded observational studies) and LOE C (low-very low; observational studies, expert opinion or other evidence). Several best practice statements (BPS) are also included, representing ungraded recommendations in which the evidence was considered to be unequivocal, although difficult to assess using the GRADE method.

The strength of recommendation was determined by the phrasing: “We recommend”, was interpreted as a strong recommendation, and “We suggest”, as a weak recommendation. A strong recommendation implied that most individuals would want the recommended intervention and should be recommended by clinicians, whereas recommendations where many individuals would not want the intervention were classified as weak.

The recommendations were classified into five categories; resuscitation, antimicrobial and source control, management and adjuvant therapy, mechanical ventilation and nutrition. This choice was made in order to provide consistency in the various areas addressed. Resuscitation included recommendations regarding indications for initial resuscitation, fluid therapy, indications for vasoactive medications, and haemodynamic assessment. The category of antimicrobial and source control includes identifying the source of the infection, indications and contraindications for antibiotic therapy and administration and recommendations for monitoring with procalcitonin. Recommendations for corticosteroids, blood products, immunoglobulin, anticoagulants and glucose were categorised under management and adjuvant therapies. Recommendations for mechanical ventilation and nutrition were classified in separate categories. Ethics approval was not applicable for this study.

3. Results

A total of 96 recommendations, including different sections of the same recommendation, were extracted from the 2016 SSC guidelines and after exclusions, 87 were analysed in this report. Fig. 1 reports a flow chart selection following the PRISMA guidelines. Distribution of level of evidence of recommendations is detailed in Table 1 . The majority of recommendations were based on low-quality evidence (37 [42.5%]), followed by 28 (32.2%) based on moderate-quality, 15 (17.2%) were BPSs and only seven (8.0%) supported by high-quality evidence (detailed in Table 2 ). Despite this, more than half were reported as strong recommendations (31 [43%]). Fig. 2 details levels of evidence depending on whether recommendations were formulated as weak or strong.

Fig. 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of the study selection.

Table 1.

The proportion of LOE and strength of recommendation overall, and for each category for SSC guidelines.

	Overall	Resuscitation	Antimicrobial and Source Control	Management and adjuvant therapy	Mechanical Ventilation	Nutrition
Totala	87	21	17	23	14	12
Level of evidence
LOE A	7 (8)	2 (9.5)	0	2 (8.6)	3 (21.4)	0
LOE B	28 (32.2)	5 (23.8)	3 (17.6)	6 (26)	9 (64.3)	5 (41.7)
LOE C	37 (42.5)	10 (47.6)	7 (41.2)	11 (47.8)	2 (14.3)	7 (58.3)
BPS	15 (17.2)	4 (19)	7 (41.2)	4 (17.3)	0	0
Strength of recommendationsa (n = 72)
Strong	31 (43)	6 (35.3)	3 (30)	7 (36.8)	10 (71.4)	5 (41.7)
Weak	41 (57)	11 (64.7)	7 (70)	12 (63.2)	4 (28.6)	7 (58.3)

Data shown as n (%). LOE: Level of Evidence; BPS: Best Practice Statement.

^aFifteen episodes of BPS were excluded from strength of recommendations.

Table 2.

Recommendations based on high-quality evidence in the SSC guidelines.

Recommendations	Category	Strength of recommendation
We recommend against using hydroxyethyl starches (HESs) for intravascular volume replacement in patients with sepsis or septic shock	Resuscitation	Strong
We recommend against using low-dose dopamine for renal protection	Resuscitation	Strong
We recommend that RBC transfusion occur only when haemoglobin concentration decreases to < 7.0 g/dL in adults in the absence of extenuating circumstances, such as myocardial ischemia, severe hypoxemia or acute haemorrhage	Management and adjuvant therapies	Strong
We recommend using a target tidal volume of 6 mL/kg predicted body weight (PBW) compared with 12 mL/kg in adult patients with sepsis-induced ARDS	Mechanical ventilation	Strong
We recommend against the routine use of the PA catheter for patients with sepsis-induced ARDS	Mechanical ventilation	Strong
We recommend using spontaneous breathing trials in mechanically ventilated patients with sepsis who are ready for weaning	Mechanical ventilation	Strong
We recommend a protocolised approach to blood glucose management in ICU patients with sepsis, commencing insulin dosing when two consecutive blood glucose levels are > 180 mg/dL. This approach should target an upper blood glucose level ≤ 180 mg/dL rather than an upper target blood glucose level ≤ 110 mg/dL	Management and adjuvant therapies	Strong

Fig. 2.

The proportion of recommendations by the strength of recommendation and level of evidence.

3.1. Resuscitation

From a total of 21 abstracted recommendations regarding resuscitation, almost half (6 [35.3%]) were characterised as strong. However, only 2 of these (9.5%) were based on high-quality evidence (Table 1). These recommendations (Table 2) were “against” hydroxyethyl starches and low-dose dopamine use.

3.2. Antimicrobial and source control

Although 10 out of a total of 17 abstracted recommendations involving antimicrobial and source control were characterised as strong, none qualified as high quality due to the lack of RCT supporting them (Table 1, Table 2).

3.3. Mechanical ventilation

Fourteen recommendations for mechanical ventilation were abstracted, 10 of which were characterised as strong recommendations (Table 1). A protective ventilator strategy and using spontaneous breathing trials for weaning were high-quality recommendations, as well as a negative recommendation of monitoring sepsis-induced acute respiratory distress syndrome (ARDS) with pulmonary artery catheterisation (Table 2).

3.4. Nutrition

A total of 12 recommendations were abstracted for nutrition, categorised into 5 (41.7%) strong and 7 (58.3%) weak recommendations. Of the 5 strong recommendations, none was based on high-quality evidence (Table 1, Table 2).

3.5. Management and adjuvant therapy

Recommendations regarding management, specific measures and adjuvant therapies in critically ill patients comprised the largest group (26.4%) of SSC recommendations. Among 23 recommendations, almost half (11 [47.8%]) were based on low-very low-quality evidence (Table 1). Despite this, nearly half (7 [36.8%]) were characterised as strong recommendations (Table 1, Table 2).

4. Discussion

In the 2016 SSC guidelines, less than 10% of recommendations were supported by randomised controlled trials. Low, very-low quality evidence and expert opinion predominated in antimicrobial/source control, and nutrition recommendations. Our findings suggest that based on current evidence, defining a universally accepted standard of care is questionable. Patient-centred outcomes should be evaluated, optimally in clinical phase III trials with the relevant interventions, including complications, mortality and functional outcomes.

The Surviving Sepsis Guidelines have been adopted for use in emergency departments and intensive care units in many countries. Whilst some learning societies did not ratify the original version (e.g. ANZICS), the guidelines have been thoroughly endorsed and adopted in many institutions where all recommendations are “standard of practice”. We have reported that although half of the recommendations were qualified as strong, a significant percentage of these were based on low-very low levels of evidence (see Fig. 2, Table 1). Nevertheless, we accept that the guidelines represent a crucial tool for the bedside caregiver. These controversies are one explanation of why SSC guidelines, as other guidelines, are not widely applied in our daily practice, with a variable compliance with clinical practice guidelines identified in a 1-day audit at 66 French adult intensive care units [10].

The observations of the current project lead the way to further attempts to upgrade these important guidelines. With the GRADE method, high-quality evidence is based on RCTs without major limitations, namely on observational studies with large magnitude effects [4], [11]. It should be acknowledged, however, although RCTs might be the most appropriate process, in sepsis, with heterogeneous pathogens, host responses and clinical presentations, conducting RCTs might be difficult, affecting overall quality of evidence. A study conducted on HAP and VAP guidelines also found that less than 10% of the recommendations were linked to high-quality evidence (based on RCTs), while the majority of the recommendations relied on expert opinion and case studies [5]. On the other hand, within the intensive care unit, critically ill patients often have altered physiological parameters that represent an important limitation for generalisation of conclusions from observational cohorts in non-critically ill patients. Individual recommendations personalised to patients with septic shock need to incorporate information from pharmacokinetic/pharmacodynamic studies, which have reported that underdosing is common in the ICU setting and validated in well-designed prospective studies [12], [13]. Safety issues in the critically ill patient should be an additional concern [14], [15].

At a practical level, our findings raise concerns on the empiric use of some recommendations and emphasise the need for reconsiderations and amendments when SCC guidelines will be updated in the future. An important recommendation that needs to be reappraised for example, is the volume of suggested fluid bolus (30 ml/kg) to be administered in septic patients with differing underlying conditions (e.g. abdominal sepsis versus pneumonia) [16] or those with alveolar-capillary damage (possibility of aggravating extravascular lung water) [16]. Another example is about lactate measurement in sepsis: measurement itself should be accompanied by a suggested intervention based on each measurement so as to improve patient management [17], [18].

Whilst most of us would agree antibiotics are an important component of the treatment of sepsis, there is no mention to the heterogeneity of the syndromes [19], [20]. Blanket use of antibiotics in all (or most) presentations of hypotension predisposes to the overuse of antibiotics with the concomitant side effects and downstream consequences of antibiotic administration, not the least of which is changes in the microbiome of patients [21]. The Infectious Diseases Society of America did not endorse SSC guidelines and expressed their concerns regarding recommendations for diagnosis and antimicrobial treatment in a public statement [22].

The goal of consensus guidance is to standardise care, improving outcomes limiting variability and dangerous practices, and defining standard of care to facilitate future research. This approach can not always be addressed by clinical trials, as in antimicrobial stewardship, infection control measures or diagnostic techniques. Recent data on poor evidence supporting antimicrobial stewardship in the ICU [23] reinforce the need to take in consideration the value of expert opinion. For instance, source control is fundamental for treatment, but it is not given high level of evidence in SSC guidelines. Observational studies in intra-abdominal infections are very consistent [24], [25] on the role of source control and a randomised trial with no intervention would be unethical. However, studies using tools such as network meta-analyses [26] are needed to determine what delay in source control would be acceptable and what would drive the need for urgent surgical drainage.

Experts agree that a suggestion for an intervention should be used in the context of observational research while awaiting trial results; however, this does not replace randomised clinical trials for therapy. The risks of prescribing drugs or strategies not supported by clinical trials with comparative control groups has been addressed in a recent editorial comment regarding COVID-19 practices [27]. In presence of weak evidence to support strategies, different approaches can be developed as done by the American Thoracic Society Guidance for COVID-19 [28]. A pragmatic approach is the derivation of suggestions using a process that has been shown to create recommendations that are concordant with guideline recommendations created using Institute of Medicine adherent methodology [29], [30]. Alternatively, a process of creating recommendations using multi-criteria decision analysis (MCDA) has been adopted for conflicting areas, such as management of ventilator-associated pneumonia or identification of a global priority pathogen list of multidrug resistant bacteria in the ICU [31], [32], [33], [34].

Several limitations should be mentioned. First, the more recent guideline update included literature published until 2016. Since then, new evidence might have been published in controversial areas and evidence can be upgraded. Most CPGs are updated in periods longer than 5 years, with difficult incorporation of emerging evidence until next iteration. Published literature should be periodically reassessed to update CPGs references and recommendations, in the form of a dynamic document. Like in other guidelines, the creation of an adaptation framework for integration of new evidence in CPG [35] would avoid outdated recommendations and improve implementation in clinical practice. Second, we did not review literature supporting each recommendation and quality of evidence was not re-assessed. Analyses were based on the classification reported in the original report, with no changes in the rating assigned to the writing committee.

Despite these limitations, our study contains important strengths. It provides a detailed analysis of quality of evidence available in septic shock, remaining a meaningful assessment in evidence-based medicine. Indeed, areas in which higher quality evidence is scarce reveal research opportunities.

5. Conclusion

In conclusion, SSC recommendations were mainly supported by observational studies, case reports and expert opinion, implying controversial recommendations. To improve patient-centred outcomes further, well-designed, high-quality studies are required focusing on currently controversial areas. Guidelines based on stronger evidence will probably be more widely accepted and implemented to become a stronger tool that will complement the medical reasoning leading to improve patients’ outcomes. It is urgently needed to develop precision approaches based on high-quality clinical data. Adaptive multicentre trials with interim analysis can determine if an intervention is superior to another, modifying the standard of care quickly to test new therapies. Using multi-criteria decision analyses or a Delphi-like process, such as the Convergence of Opinion on recommendations and Evidence (CORE) process (using Institute of Medicine adherent methodology), are potential tools in making clinical recommendations in areas of weak evidence. This is particularly important because within the SSC guidelines most judgements were informed by indirect evidence and non-systematic observations. All suggestions need to be reconsidered as new evidence accumulates.

Disclosure of interest

JR served in the speaker's bureau or consultant for Pfizer, MSD and Astellas. The other authors declare that they have no competing interest.

Funding

This work was supported in part by a grant from the Observationship Programme (EX) from the European Society of Clinical Microbiology & Infectious Diseases.

Compliance with ethical standards

Ethics committee approval was not required.

Author contributions

JR and JFC designed the study. The recommendations were abstracted by a single reviewer (EX) and validated by another reviewer (ST). CSL and LC revised the methodology. EX analysed the collected data. JR, ST, and EX wrote the first draft of the manuscript. DK and JL contributed scientifically in subsequent drafts. All authors approved the final version of the manuscript. EX takes full responsibility for the integrity of reported data.