Why we need a new definition of sepsis

Abstract

On April 23, 2015, Kaukonen and colleagues published an article in the New England Journal of Medicine entitled “Systemic inflammatory response syndrome criteria in defining severe sepsis”, which investigated the sensitivity and validity of using SIRS criteria to define intensive care unit (ICU) patients with severe sepsis. This study used admission data of over 100,000 patients in order to investigate patients with severe sepsis who either met or didn’t meet SIRS criteria. The investigators found that in-hospital mortality increased linearly with the number of SIRS criteria met; raising concern that SIRS criterion is not sensitive enough. This study of SIRS criteria raises important questions about the recognition and diagnosis of severe sepsis.

The systemic inflammatory response syndrome (SIRS) was described by the American College of Chest Physician and Society of Critical Care Medicine in a consensus statement from 1991 as part of a larger effort to uniformly define sepsis (1). The aim of this recently published study by Kaukonen and colleagues, “Systemic inflammatory response syndrome criteria in defining severe sepsis”, was to assess the sensitivity and validity of using SIRS criteria for this purpose. This study evaluated patients whose clinical presentation suggested severe sepsis, comparing those who met SIRS criteria to those who did not meet SIRS criteria. The study was a remarkable effort involving review of over 1 million patients cared for in Australian and New Zealand intensive care units (ICUs) from 2000 to 2013, accounting for approximately 90% of all ICU admissions in this area during this time (2). The primary outcome was in-hospital mortality, with a secondary outcome being place of discharge (home, rehab or other hospital). The hypothesis was that there would be a linear increase in the risk of death, not a defined transition point after two criteria (the definition of meeting SIRS criteria). In studying over 100,000 septic patients, the investigators found no real transitional increase at two criteria, which raises questions on the sensitivity and validity of using SIRS to define severe sepsis.

When SIRS criteria were initially defined more than 20 years ago, the goal was to provide a “practical framework” for use in clinical practice as well as in research settings (3). Prior to these definitions, there was limited uniformity to sepsis definitions used across research teams, leading to difficulty with generalizing findings (4). Criteria for SIRS included specific changes in body temperature, heart rate, tachypnea or hyperventilation and white blood cell count, with two or more of these being necessary to label the patients with SIRS. Sepsis was defined as a subcategory of SIRS patients who had a documented or suspected source of infection. Severe sepsis narrowed this category to patients with organ dysfunction, and septic shock was a subcategory of severely septic patients with hypotension. These criteria and thresholds were chosen by expert consensus, with the goal to have some standardization across medical centers and research groups. Data at the time showed higher risk of mortality for patients meeting these criteria on ICU admission (1). These definitions of SIRS, sepsis and septic shock has been used clinically and throughout research studies for the past few decades, but have evoked considerable controversy (4-9). SIRS criteria was a clinical syndrome description, and as such may combine several distinct pathophysiological pathways (5,9). Septic patients who do not fulfill SIRS criteria may be excluded from sepsis investigations, and may receive a delay in appropriate treatment. Kaukonen and colleagues have made a significant contribution by investigating the clinical outcomes for this group of patients who would otherwise be excluded by the SIRS definition.

This study aimed to evaluate sensitivity, face validity and construct validity of SIRS. Sensitivity is the ability of a test to recognize true positives, while specificity measures number of true negatives correctly identified. In a screening test for a potentially life-threatening disease, such as severe sepsis, high sensitivity would be valued over high specificity so that cases are not missed. An important concern raised by this study is that one in eight patients with sepsis is missed by the SIRS criteria, indicating an undesirably low sensitivity. Investigators have also criticized SIRS criteria for the lack of ability to differentiate between septic and non-septic patients (poor specificity) (10). The face validity refers to the transparency or relevance of a test as it appears to test participants, i.e., that the test looks like it is going to measure what it’s supposed to measure (11). Examining the face validity requires some idea of what those using the test believe it should show. Construct validity indicates the degree to which a test measures what it purports to measure. Although the study did not explicitly study or quantify how much the SIRS criteria contribute to making a diagnosis of sepsis, it is a reasonable inference that many critical care clinicians use SIRS criteria in their diagnosis of a septic patient. However, SIRS criteria are not required for a diagnosis of sepsis (contrary to the 1991 consensus definition), as some patients were labeled as SIRS-negative and simultaneously identified by clinicians as having sepsis. SIRS was not designed to measure illness severity or short-term sepsis mortality but was designed to be exquisitely sensitive in not missing patients with sepsis, and therefore is lacking in construct validity.

A central limitation of all studies of severe sepsis is that there is no accepted gold standard for a definition of severe sepsis. Kaukonen’s study, although excellent, is not immune to this limitation. The designation of severe sepsis was limited to information obtained in the first 24 hours of ICU admission and based on coding at that time: severe sepsis was defined as having APACHE III diagnoses of infection plus at least one organ failure or APACHE III diagnoses of severe sepsis or septic shock. Therefore, the diagnosis of sepsis in this study is really a definition based on coding and APACHE III diagnoses, a method which may have inherent limitations, much like the 1991 consensus definition that relies on SIRS.

Patients labeled as SIRS-positive severe sepsis met two or more SIRS criteria in addition to these criteria for severe sepsis, while SIRS-negative severe sepsis met less than two SIRS criteria. Pneumonia, gastrointestinal rupture, and biliary infection were common diagnoses (18.2%, 18.5% and 10.4%, respectively) among the SIRS-negative patients. Of SIRS-negative patients, 20% (n=2,624) did not meet any SIRS criteria. This group had a high proportion of patients with septic shock (33%, n=866) or mechanical ventilation (51%, n=1,329). Although these proportions may seem high, they represent a very small percentage of all patients. Only 0.8% of patients with septic shock and 1.2% of patients with mechanical ventilation had zero SIRS criteria. Taken together, the data suggest that clinicians are more likely to diagnosis a SIRS negative patient with sepsis if they have severe organ failure, such as shock or respiratory failure, or if there is evidence of a disease that is highly associated with infection. Like all studies that rely on clinical registry surveillance, data were gathered by collectors in the ICU as part of a routine process, which is by design susceptible to missing information as well as misclassification. However, individual validation of whether all 1.2 million patients were appropriately categorized is infeasible. Similarly, in a study of this magnitude, there is no feasible mechanism by which one could identify all patients that were incorrectly excluded from the study.

It is difficult to identify patients with severe sepsis in a way that allows classification for both clinical care and research purposes. Reliance solely on SIRS criteria may be insufficiently sensitive, and is certainly not specific. Therefore, there may be value in using a screening test for sepsis (highly sensitive) and a confirmatory test (highly specific). SIRS has never been very specific (12) and was not designed to be so. Sepsis and SIRS criteria have been reevaluated by the Surviving Sepsis Campaign and the diagnostic criteria for sepsis were significantly expanded to include an extensive list of other indicators of infection, inflammation, hemodynamic abnormalities or organ dysfunction (13-16). The myriad indicators of sepsis in the revised definition may increase sensitivity, but the need for specificity remains unfulfilled. As a response, several biomarkers have been investigated for use in confirming the diagnosis of sepsis, including procalcitonin, C-reactive protein, tumor necrosis factor-α, various interleukins and protein C (7). Procalcitonin may have the most utility for identifying an infectious cause of SIRS (7,17) and there have been suggestions for using procalcitonin levels to classify sepsis and grade severity (18). Testing to this point indicates that procalcitonin is more sensitive and less specific, depending on cutoff values used (19,20). However, these biomarkers remain investigational, and have yet to be validated sufficiently for widespread clinical use.

We believe that future directions for improving identification of sepsis may rely on more complex quantification. The SIRS definition is simple clinically and is an easy set of inclusion criteria for sepsis research (21-23). However, the clinician, when diagnosing sepsis, is likely subconsciously applying a Bayesian algorithm that includes the SIRS criteria as well as several other clinical parameters, such as severity of disease (for example, giving more weight to a white blood cell count of 21,000 vs. 11,000 cells/µL) or known diseases that confer a high probability of sepsis (i.e., intestinal perforation). A complex algorithm that attempts to recognize this clinical syndrome may prove superior to more conventional definitions. Several other scoring systems have improved accuracy by weighting continuous vs. dichotomous data, such as the eCURB vs. the CURB-65 for scoring pneumonia (24). Complex Bayesian scoring systems would require a computer, ideally, to quickly acquire and process several clinical data from the patient’s record and produce a diagnostic probability of sepsis. The data acquired from Kaukonen and colleagues’ study is an excellent resource that could be used to develop and test such systems.

The results of Kaukonen and colleagues’ study demonstrate that the SIRS criteria are flawed in recognizing sepsis. The results of this study will likely not directly change clinical practice, as clinicians are already diagnosis sepsis by rules that differ from the 1991 consensus statement, which implies that there is already recognition among clinicians that the SIRS criteria are limited. Perhaps some clinicians, upon reading this study, will be reminded that patients who do not meet SIRS criteria still may have significant morbidity and mortality. However, the true value of this study is the insight it affords in future leaders of critical care in designing new criteria for recognition of sepsis. This study should be a call to arms that critical care physicians and hospitals need to develop a better screening tool than the current one. The future diagnostic method will likely employ large, multidimensional clinical data obtained from the medical record and Bayesian algorithms to arrive at an improved determination of sepsis.