NICE guidance on sepsis is of limited value in postoperative colorectal patients: the scores that cry ‘wolf!’

Abstract

Background and aims

Late recognition of sepsis and consequent death remains a problem. To address this, the National Institute for Health and Care Excellence has published updated guidance recommending the use of the Quick Sequential Organ Failure Assessment (Q-SOFA) score when assessing patients at risk of sepsis following the publication of the Third International Consensus Definitions for Sepsis and Septic Shock. The trauma from major surgery produces a systemic inflammatory response syndrome (SIRS) postoperatively as part of its natural history, which may falsely trigger scoring systems. We aimed to assess the accuracy of Q-SOFA and SIRS criteria as recommended scores for early detection of sepsis and septic complications in the first 48hrs after colorectal cancer surgery.

Methods

We reviewed all elective major colorectal operations in a single centre during a 12-month period from prospectively maintained electronic records.

Results

One hundred and thirty nine patients were included in this study. In all, 29 patients developed postoperative infective complications in hospital. Nineteen patients triggered on SIRS without developing infective complications, while 42 patients triggered on Q-SOFA with no infective complications. The area under the ROC curve was 0.52 for Q-SOFA and 0.67 for SIRS.

Discussion

Q-SOFA appears to perform little better than a coin toss at identifying postoperative sepsis after colorectal cancer resection and is inferior to the SIRS criteria. More work is required to assess whether a combination of scoring criteria, biochemical markers and automated tools could increase accurate detection of postoperative infection and trigger early intervention.

Introduction

Postoperative infectious complications pose a significant challenge in the care of surgical patients. Postoperative sepsis is reported in 4.3% of patients having major cancer surgery, with 30-day mortality in these patients ranging from 0.5% in low-risk groups to 5.5% in high-risk patients.¹ Postoperative sepsis is also associated with increased mortality for up to one year after surgery.² Early recognition and intervention in cases of sepsis may minimise morbidity and mortality.^3–5

A number of clinical scoring tools have been developed to aid early recognition of sepsis and therefore early intervention. The Third International Consensus Definitions for Sepsis and Septic Shock has recommended that a new scoring system, the Quick Sequential Organ Failure Assessment (Q-SOFA) be used to help bedside diagnosis of sepsis.⁶ This scoring system replaced the systemic inflammatory response syndrome (SIRS) criteria, first described in 1992 and used in previous sepsis guidelines.^4,7 Changing from the SIRS criteria to Q-SOFA places the emphasis on organ dysfunction rather than physiological manifestations of inflammation. This change was incorporated into the 2016 National Institute for Health and Care Excellence (NICE) guidance for management of sepsis,⁸ with a few additions and this guideline also goes on to recommend that any patient at moderate to high risk of death from sepsis should be reviewed by a senior decision maker (i.e. a doctor of Speciality Training year 3 or above) within one hour of concerns being raised.⁸

While scoring systems may aid prompt initiation of treatment for sepsis,⁵ these scores may also be falsely triggered by patients following elective surgery who do not have sepsis.⁹ It is well known that the stress response to trauma or surgery leads to activation of inflammatory cytokines, which may lead to SIRS in the absence of infection.^10–12 SIRS occurs in 77.8–94.3% of patients following gastrointestinal surgery,¹³ of whom 99.1% recover without the development of severe organ dysfunction.¹³

As SIRS is an almost ubiquitous feature of the postoperative course, without complication in the vast majority of cases, triggering of sepsis early warning tools by a SIRS caused by the expected trauma of surgery may lead to inappropriate administration of broad-spectrum antimicrobials. This in turn has the potential to increase antibiotic resistance¹⁴ and exposes patients to the risk of hospital acquired Clostridium difficile infection.¹⁵ As it has been shown that greater antibiotic stewardship can significantly decrease the incidence of these infections and so prevent patients from developing serious iatrogenic infections,¹⁶ it is vital that antibiotics are administered only when necessary.

In view of the change in NICE and international sepsis guidance, we aimed to assess both the SIRS and Q-SOFA scoring tools and their predictive value in detecting postoperativee infective complications in a cohort of patients who had elective colorectal surgery.

Methods

A consecutive series of patients who had elective major colorectal cancer operations at one institution during a 12-month period was identified retrospectively using an electronic database search. This study was registered with the hospital’s audit department (SB-Gen-2016/17-448). Case notes were manually searched and the postoperative course of each patient was reviewed, including observations, blood results and complications. SIRS or Q-SOFA trigger criteria were recorded for the first 48 hours after surgery. Electronic prescribing records were used to determine the number of patients who were treated with antibiotics. Postoperative complications occurring to 30 days postoperatively, both infective and noninfective, were stratified using the Clavien–Dindo classification.¹⁷ Infective complications were defined as the need for the prescription of antibiotics or the drainage of pus.

The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for SIRS and Q-SFOA to predict postoperative infectious c omplications were then calculated with ROC curves. Statistical analysis was performed using SPSS version 22 (IBM, NY, USA).

Results

The postoperative course of 139 patients, 85 of whom were male (61%) having elective surgery for colorectal cancer was reviewed. The median age of the cohort was 72 (interquartile range, IQR, 64–77) years. Table 1 describes the operations performed. Sixty-one (44%) operations were open, 54 (39%) were completed laparoscopically, 11 (8%) were converted from laparoscopic to open and 13 (9%) were carried out robotically.

Table 1.

SIRS and qSOFA criteria (from Singer et al. 2016).⁶

SIRS (2 or more of)	qSOFA (2 or more of)
Temperature > 38°C or < 36°C	Respiratory rate > 22 beats/minute
Respiratory rate > 20 beats/minute	Altered mentation
Heart rate > 90 beats/minute	Systolic blood pressure < 100 mmHg
White blood cell count > 12 × 109/l or < 4 × 109/l

Of the 139 patients, 34 (24%) triggered two or more of the SIRS criteria at one point or more during the first 48 hours postoperatively. Fifty-four (39%) patients triggered the Q-SOFA criteria at some point within the first 48 postoperative hours. Eighteen (13%) patients triggered both SIRS and Q-SOFA criteria. Of the 139 patients in the study, 29 (21%) went on to develop infectious complications during their recovery. Table 2 shows sensitivities, specificities PPVs and NPVs for SIRS and Q-SOFA. Figure 1 displays ROC curves for SIRS and Q-SOFA. The area under the ROC curve for SIRS was significantly greater than that under the curve for Q-SOFA: 0.67 (95% confidence interval, CI, 0.55–0.97) compared with 0.52 (95% CI 0.40–0.64).

Table 2.

Operations performed.

Operation	Procedures (n)
Right hemicolectomy	53
High anterior resection	38
Low anterior resection	15
Abdominoperineal resection	11
Ultra-low anterior resection	7
Defunctioning loop colostomy	5
Subtotal colectomy	3
Completion colectomy	2
Hartmann’s procedure	2
Extended right hemicolectomy	1
Sigmoid colectomy	1
Panproctocolectomy	1
Total	139

Figure 1.

ROC curve for SIRS and Q-SOFA as predictors of postoperative infective complications

Table 3.

The performance of SIRS and Q-SOFA as predictors of the development of infectious complications in the early postoperative period (0–48 hours).

	SIRS	Q-SOFA
Specificity	0.83	0.62
Sensitivity	0.52	0.41
Positive predictive value	0.44	0.22
Negative predictive value	0.87	0.80
Area under ROC curve	0.67	0.52

In all, 19/139 patients triggered two or more SIRS criteria without developing an infectious complication, while 42/139 patients triggered Q-SOFA without developing an infectious complication. A total of 45 complications occurred. There were 25 Clavien–Dindo class I complications, 17 class II complications and 3 class III complications. The 17 class II complications comprised entirely postoperative complications requiring therapy with antibiotics. There were three patients with anastomotic leaks, all of whom returned to theatre, and there were no inpatient deaths.

Discussion

Both SIRS and Q-SOFA applied in the first 48 hours postoperatively are poor at predicting infective complications after elective surgery for colorectal cancer and fail to accurately discriminate between the normal postoperative course and an inflammatory response due to infection. The change proposed in the NICE and international sepsis guidelines from SIRS criteria to Q-SOFA should not be applied to elective surgical patients in the immediate postoperative period, as Q-SOFA was significantly worse than SIRS at discriminating between normal physiology and infection. Simplistic scoring systems are attractive in that they are intended to facilitate quick assessment by junior members of the nursing and surgical teams; however, they are intended for use in the context of patients with suspected infections, which is not to be confused with ‘any patient with a fever’.

Q-SOFA was designed for use across a wide variety of patients with a range of pathologies and was not designed specifically for the postoperative surgical patient. Indeed, some studies investigating the utility of Q-SOFA exclude elective surgical patients in order not to influence their results.¹⁹ Q-SOFA was originally intended for use as a predictor of mortality in patients with sepsis and not as an early warning screen for sepsis.²⁰ Despite this, following the publication of Sepsis 3,⁶ the recommendation to use Q-SOFA was adopted by NICE in the UK.⁸

Previous work in other settings had validated Q-SOFA as a better predictor of sepsis than other scoring systems such as SIRS;²¹ however, this work was carried out from 1.3 million electronic health records of a huge spectrum of patients with a large amount of clinical heterogeneity. Q-SOFA has also been shown to perform better than SIRS in a cohort of critically ill patients requiring ICU admission,²² although evidence for its use postoperatively is lacking.

SIRS has been in use for a number of years in UK practice and high scores have been shown to be associated with poorer outcomes.²³ However, SIRS has previously been shown to be an almost ubiquitous part of the normal postoperative course in the first 48 hours following surgery, with previous work suggesting that it becomes a useful discriminator from the third postoperative day.¹⁸ This is probably due to the increasing incidence of the most significant postoperative infectious complications such as anastomotic leakage, intra-abdominal collection or major wound infections, together with their associated physiological derangements after this time.

Other authors from non-surgical backgrounds have questioned the universal application of Q-SOFA, with concerns being raised about the impact of changing from an already well-known and well-performing system (SIRS)²⁴ and the potential lack of specificity of Q-SOFA, which can classify up to one-third of acute admissions, especially in the elderly, as septic, leading to overburdening of senior decision makers.²⁵

In addition to its poor performance as a screening tool, Q-SOFA also is slower than SIRS in identifying sepsis requiring intensive care admission.²⁶ This may lead to poorer patient outcome, given the importance of early detection and treatment of severe sepsis.

Although superior to Q-SOFA, SIRS scoring still remains a poor early warning tool for postoperative infection. With the introduction of the national early warning score (NEWS) in the UK²⁷ and the increasing adoption of electronic patient observation systems in many hospitals,^28,29 it is possible to track physiological parameters that may improve the identification of acutely ill patients by observation of trends and triggering of NEWS score alert.^27,30 These have been shown to be good indicators of possible sepsis in surgical patients.³¹ Potential combination of scoring systems and automated screening tools may therefore lead to early detection and prompt treatment of sepsis.³²

The limitations of this study are its small sample size and retrospective nature. These limitations were, however, minimised by our centre’s use of electronic observation and prescription systems, which facilitated as close to 100% data completeness as possible.

Conclusions

Q-SOFA appears to perform little better than a coin toss at identifying postoperative sepsis after colorectal cancer resection in the first 48 hours postoperatively and is inferior to SIRS criteria. Further, the use of any simplistic scoring systems to assess patients in the initial postoperative period is unhelpful, owing to the low incidence of infective complications and the expected derangement of physiological parameters expected from the normal surgical stress response. More work is required to assess whether a combination of scoring criteria, biochemical markers and automated tools could increase accurate detection of postoperative infection and trigger early intervention.

Acknowledgement

We wish to thank Jane Heeley, cancer audit team leader at the Royal Derby Hospital, for her help with case identification and Anne Horne, clinical secretary at the Royal Derby Hospital, for her help with record retrieval. Philip Herrod is supported by a research training fellowship jointly awarded by the Royal College of Surgeons of England and the Dunhill Medical Trust.