Sepsis criteria versus clinical diagnosis of sepsis in burn patients: a validation of current sepsis scores

Abstract

Background:

Sepsis remains an ongoing diagnostic challenge in burns, especially with the signs of sepsis being ubiquitously present during the acute period after injury. We aimed to determine the predictive validity of three current sepsis criteria in the burn population. The criteria of interest included the American Burn Association (ABA) sepsis criteria, Mann-Salinas et al. predictors of sepsis, and the Sepsis-3 consensus definition.

Methods:

Adult patients with an acute burn injury who were diagnosed prospectively with sepsis by the burn team using specific clinical markers were included in this cohort study. Sepsis predictors were collected from patient charts and used to calculate the results of the three criteria, then subsequently compared to the clinical diagnosis.

Results:

Eighty-eight out of 418 (21%) patients in the study (2006–2016) were septic; mean age was 50 ± 18 years with a mean percent total body surface area (TBSA) burn of 30% ± 17%. Inhalation injury was present in 50%, median length of stay was 49 (29 – 71) days, and mortality was 19%. The ABA, Mann-Salinas, and Sepsis-3 criteria were positive in 59%, 28%, and 85% respectively, p<0.05. The most reliable predictors included increased oxygen requirements, altered mental status, hypothermia/hyperthermia, tachycardia, and hypotension.

Conclusion:

The Sepsis-3 criteria was the most predictive, followed by the ABA and Mann-Salinas criteria. However, no criterion alone had the accuracy to be a diagnostic standard within this burn population. We recommend sepsis is clinically assessed, diagnosed, and documented prospectively by the burn team, and not by the application of retrospective criteria.

INTRODUCTION

Diagnostic Challenges in Burn Sepsis

One of the major contributors to morbidity and mortality in burn patients is sepsis.¹ Sepsis stems from a potentially life-threatening infection with systemic manifestations and can progress to septic shock, which is defined by hyper-dynamic cardiovascular instability. This can result in subsequent tissue hypo-perfusion, resulting in multi-organ failure, increased hypermetabolic catabolism, impaired wound healing, and trigger inflammatory and stress responses leading to immune exhaustion and failure.^{1, 2} These events ultimately lead to a substantially increased mortality.

In general, burn patients are uniquely susceptible to develop infection and sepsis, particularly in patients with burn injury involving a large percentage of their total body surface area (%TBSA).^3–7 Even though it is well established that burn patients are innately prone to infections and sepsis, the diagnosis of sepsis in this population proves particularly challenging as the systemic response to burn injury itself clinically mimics that of sepsis. The post-burn systemic response leads to an increase in the patient’s core body temperature, heart rate, respiratory rate, and hyperglycemia, most of which are used as predictors for the definition of sepsis and the systemic inflammatory response syndrome (SIRS).^{1, 2, 8–10} As a result, the traditional SIRS and sepsis criteria are unreliable in most burn patients.¹¹

Current Sepsis Criteria

To address the need for burn-specific diagnostic criteria, the American Burn Association (ABA) developed the burn-specific sepsis definition in 2007 to standardize the definition of sepsis in this patient population.¹² A clinical suspicion of infection, in addition to meeting at least 3 of the 6 burn-specific predictors, should prompt an investigation for the infectious source and initiation of empiric systemic antibiotics (Table 1).

Table 1.

Sepsis criteria evaluated.

American Burn Association (ABA) Sepsis Criteria12	Mann-Salinas et al.11 novel predictors of sepsis:	Sepsis-3 Consensus Definition for Sepsis and Septic Shock13
The trigger includes at least three of the following: Temperature >39° or <36.5°C Progressive tachycardia (>110bpm) Progressive tachypnea Thrombocytopenia Hyperglycemia Inability to continue enteral feedings 24 hours	Tachycardia > 13 0 bpm Mean Arterial Pressure <60 mmHg Base deficit <−6 mEq/L Hypothermia <36°C Use of vasoactive medications Hyperglycemia > 150 mg/dL	qSOFA Score: Suspected or documented infection and qSOFA ≥2 and/or SOFA >2. Altered mental status (Glasgow Coma Scale <13) Systolic blood pressure ≤100 mmHg Respiratory rate ≥22 breaths/min
Infection, at least one of the below: Culture positive infection, or Pathologic tissue source identified, or Clinical response to antimicrobials		SOFA Variables: PaO2/FiO2 ratio Glasgow Coma Scale Mean Arterial Pressure Vasopressor requirements Serum creatinine or urine output Bilirubin Platelet count
		Septic Shock: Vasopressors required to maintain MAP >65 mmHg Lactate >2 mmol/L (after adequate fluid resuscitation)

ABA, American Burn Association; SOFA, Sequential Organ Failure Assessment; qSOFA, quick SOFA. BPM, beats per minute

Although the ABA sepsis criteria have since been considered the prototypical criteria in diagnosing sepsis in thermally injured patients, its limitations became apparent in 2013 when a retrospective, case-controlled study by Mann-Salinas et al. proposed six novel burn-specific sepsis predictors that were found to have improved predictive ability compared to the ABA sepsis criteria (Table 1).¹¹ This novel criterion introduced predictors that assess cardiovascular status, which is of importance when monitoring progression to septic shock. This novel criterion proposed by Mann-Salinas et al., however, has yet to be validated in larger studies.

In 2016, the definition of sepsis and septic shock in critically ill patients was updated with the Third International Consensus Definition for Sepsis and Septic Shock (Sepsis-3).¹³ This definition is based on the use of the Sequential Organ Failure Assessment (SOFA) (Table 1). This burn-nonspecific Sepsis-3 criteria has yet to be evaluated in the burn patient population.

Objective of Study

Despite these existing definitions, our group hypothesized that the use of a single criteria in isolation is not predictive enough to prospectively diagnose septic patients. We believe a prospective and clinical diagnosis by the burn team and immediate documentation is the best approach to identify sepsis. We have implemented prospective documentation of sepsis since 2006. We now have a sufficient pool of patients to evaluate the predictive validity of the criterion-based sepsis definitions compared to real-time diagnosis and documentation of sepsis. In our study, we used predictive validity, defined as the performance of a test when compared against a reference model, to study the three sepsis criteria. As sepsis lacks a definitive definition, this reference standard for our clinical diagnosis is based on clinical evidence including positive bacterial cultures, pressor usage, temperature fluctuations, alterations in leukocytes, platelets, and glucose levels, in combination with each patient’s overall clinical picture. The diagnosis was documented and flagged prospectively so the performance of the three sepsis criteria can be evaluated. Thus, the aim of this study was to compare the predictive validities of the ABA sepsis criteria, the novel predictors of burn sepsis from the 2013 Mann-Salinas et al. study (Mann-Salinas), and the Sepsis-3 criteria to a prospective clinical diagnosis.

METHODS

This project was a single institution cohort study including acute burn patients admitted to a tertiary burn centre from 2006 to 2016 with a TBSA ≥ 10%. A total of 418 patients were identified and out of those, 88 patients (22%) were diagnosed with sepsis during their acute hospitalization period. Clinical diagnoses were made prospectively by burn specialist surgeons and critical care physicians with consideration of each patient’s entire clinical picture. All patients must have had a documented or suspected infection coupled with signs of systemic manifestations of sepsis. These signs include tachycardia, hyperglycemia/hypoglycemia, enteral feeding intolerance, wound infections, alterations in temperature, radiologic or microbiological features of infectious source, vasopressor dependency, hemodynamic instability, need for active resuscitation, and administration or response to systemic antibiotics. Documentation in the patient medical records by the responsible physician during rounds based on the aforementioned parameters were used to identify clinically diagnosed septic patients for this study.

As sepsis consists of a constellation of findings with no single clinical finding being diagnostically definitive, its diagnosis can be difficult to establish. Nevertheless, we believe a prospective clinical diagnosis from burn-specialist clinicians can provide a reliable reference standard to compare with the ABA sepsis criterion, Mann-Salinas novel predictors, and Sepsis-3 definition. Medical records of the prospectively identified septic patients were subsequently reviewed to determine whether the ABA sepsis criteria, Mann-Salinas predictors and Sepsis-3 definition could identify sepsis in these patients based on their respective criteria (Table 1). Any evidence of infection (positive cultures, pathologic tissue source, response to antibiotics), along with the cumulative presence of the twelve burn-specific predictors from the ABA and the Mann-Salinas criteria, as well as an evaluation of organ failure based on the Sepsis-3 definition, were documented from the 48-hour period prior to and up to the time of the clinical diagnosis of sepsis as was noted on the patient records. Days after the documentation of the clinical diagnosis were not included as the initiation of treatment immediately post-diagnosis could potentially mask the presence of these findings. The determination of the three criteria under investigation was therefore done solely through the 48-hour window leading up to the documented time of the clinical diagnosis and initiation of treatment.

In addition, we further stratified our burn patients population into two groups based on the percentage of their severity, 10–20% TBSA burns (moderate) and ≥20% TBSA burns (severe), in order to study whether the severity of burn size affected the predictive validity of each of the three scores. The chi-square test was used to compare the predictive validities between the two criteria. McNemar’s test was used to compare the validities of each criterion within each burn severity group. All tests were declared statistically significant if p<0.05.

RESULTS

Patient Cohort

Out of 418 burn patients meeting eligibility criteria admitted to our burn centre between 2006 and 2016, 88 patients (21%) were prospectively diagnosed with sepsis during their admission in the burn ICU (Table 2). Among septic patients, mean age was 50 ± 18 years and mean burn size was 30 ± 17% TBSA. Inhalation injury was present in half of the patients (n=44). Median length of stay was 49 (29–71) days, with a length of stay per %TBSA burn among survivors overall of 1.8 days. The mortality rate for all septic patients was 19% (n=17) (Table 2). Of these septic patients, 33 patients had a %TBSA burn between 10–20% and 55 had %TBSA burn ≥20% (Table 2). Mean age of the 10–20% TBSA cohort was 53 ± 20 years and in the ≥20% TBSA group, mean age was 48 ± 17 (p=0.212). There were significantly more patients with an inhalation injury in the ≥20% TBSA group when compared with patients in the 10–20% TBSA group (60% vs. 33% respectively; p=0.027). As expected, patients with more severe burns had longer length of stays, median LOS 64 (37–81) days, than patients with less severe burns, median LOS 33 (22–46) days (p=0.005). In contrast, length of stay per percent burn among patients with severe burns was lower, median 1.6 (1.1–2.4), when compared with patients that had smaller burns, median 2.1 (1.5–3.7) (p<0.0001). Surprisingly, there was no difference in mortality between the injury severity groups (p=0.266; Table 2).

Table 2.

Demographics, injury characteristics and outcomes in septic patients.

	All	10–20%	≥ 20%	P
		TBSA	TBSA
No. of patients	88	33	55
Age, years, mean ± SD	50 ± 18	53 ± 20	48 ± 17	0.212
Male, no. (%)	71%	21 (63%)	41 (75%)	0.337
TBSA, %, mean ± SD	30 ± 17	14 ± 3	39 ± 15	<0.0001
3rd TBSA, %, mean ± SD	20 ± 18	6 ± 6	28 ± 181	0.0001
Inhalation injury, no. (%)	44 (50%)	11 (33%)	33 (60%)	0.027
LOS, days, median (IQR)*	49 (29–71)	33 (22–46)	64 (37–81)	0.005
LOS/TBSA, days/%, median	1.8 (1.3–3.1)	2.1 (1.5–3.7)	1.6 (1.1–2.4)	<0.0001
(IQR)*
Mortality, no. (%)	17 (19%)	4 (12%)	13 (24%)	0.266

TBSA, total body surface area; LOS, length of stay;

^*Analysis restricted to patients alive until discharge.

Criteria Comparison

The predictive validity for each criterion was evaluated by determining how frequently criteria-based diagnosis was positive in the patient with a clinical diagnosis of sepsis. In comparing the different definitions to real-time documentation of sepsis, we have found that the predictive validity of the Sepsis-3 definition was significantly higher than either the ABA or Mann-Salinas criteria (p<0.05). Using the Sepsis-3 definition we could predict 85% (n=75) of patients that became clinically septic 48 hours prior to the prospective diagnosis. Of those 85% (n=75) patients diagnosed as septic by the Sepsis-3 definition, 13% (n=10) met the Sepsis-3 criteria of septic shock. The ABA sepsis criteria had an overall predictive validity of 59% (n=52) and was significantly more reliable than the Mann-Salinas predictors that was found to have the lowest predictive validity, with an overall predictive validity of 28% (n=25) (Figure 1).

Figure 1.

Predictive validity of sepsis criteria by burn size. TBSA, Total body surface area; ABA, American Burn Association.

In comparing the performance of the three criteria based on the patients’ %TBSA burn, we found that for patients with 10–20% TBSA burn, the predictive validities of the ABA, Mann-Salinas, and Sepsis-3 were 58% (n=19), 27% (n=9), and 82% (n=27) respectively. For patients with ≥20% TBSA burn, the predictive validities of the ABA, Mann-Salinas, and Sepsis-3 respectively were 60% (n=33), 29% (n=16), and 87% (n=48). Our analysis showed no statistically significant differences between the two TBSA groups with respect to the validities of all three criteria.

When individual predictors were examined from all three criteria, the ones that most strongly correlated with a clinical diagnosis of sepsis in descending order were: respiratory rate >22 (Sepsis-3), altered mental status (Sepsis-3), progressive tachypnea (ABA), temperature variations (ABA), tachycardia (ABA), and systolic blood pressure <100 mmHg (Sepsis-3) (Figure 2). Other predictors, including all six from the Mann-Salinas et al., had predictive validities below 50% and were not reliably associated with sepsis in this study.

Figure 2.

Predictive validity of individual variables of sepsis criteria. American Burn Association (A), Mann-Salinas et al. (B), and Sepsis-3 (C). ABA, American Burn Association.

DISCUSSION

Burn infection and sepsis accounts for a vast amount of morbidity and mortality in burn patients.^{4, 14} Given the potential for life-threatening sequelae, physicians must remain hyper-vigilant when screening for sepsis in patients with thermal injuries. It seems that early prediction and identification of sepsis with the initiation of adequate interventions are key to improving mortality. We hypothesized that no ideal definition exist that will predict sepsis reliably in burn patients. To evaluate currently used sepsis predictors, we compared three definitions with potential applicability to burn patients and compared their predictive validities to a prospective diagnosis of sepsis.

In our current study, the 2016 Third International Consensus Definition for Sepsis and Septic Shock (Sepsis-3) significantly outperformed both burn-specific criteria for being the most reliable criteria for sepsis in our patient population. Despite being developed for use in the general ICU setting, the Sepsis-3 criteria has been shown to be the most reliable screening tool in the burn patient population in our study. By using clinical status markers documented from 48 hours prior to a prospective diagnosis, we have shown that this screening criterion can reliably predict the evolution of early sepsis days before the clinical diagnosis was established. Nevertheless, given the many complexities of sepsis and the possibilities of false positives secondary to the post-burn pro-inflammatory state, no single sepsis criterion is strongly predictive. The patient’s entire clinical picture should be considered, as opposed to focusing in on individual predictors or criteria.

The ABA sepsis criteria was found to be predictive of sepsis in about 50% of the cases. Previous studies have noted similar outcomes using the ABA criterion.^{11, 15} In its current state, the ABA sepsis criterion for adult burn patients is not reliable enough for use in clinical practice. Further revisions are needed to improve its overall reliability for sepsis. The Mann-Salinas et al. predictors performed the poorest amongst the three criteria that were studied with prediction and identification of sepsis in less than a third of cases. These predictors share similarities with the Sepsis-3 septic shock criteria as it includes clinical predictors for hemodynamic instability and hypo-perfusion. As such, it may be more suited for the diagnosis of more progressive septic episodes, as opposed to the detection of the early signs of sepsis - where it lacks the predictability to do so.

The individual markers from all three sepsis criteria that were most predictive with regards to burn sepsis included increased oxygen requirements, altered mental status (Glasgow Coma Scale <13), hyperthermia, and progressive tachycardia. Alone, these predictors are nonspecific; however, in a burn patient with an underlying suspicion of infection, the presence of two or more of these factors should strongly raise concern for early signs of sepsis.

Sepsis is known to be a complex syndrome involving the intersection of numerous clinical factors.^{13, 16} There are no objective clinical, laboratory, or radiographic indicators that represent the gold standard for the diagnosis of burn sepsis. In our study, we calculated the predictive validity of each criterion using a prospective clinical diagnosis as a reference model for the presence of sepsis. Unfortunately, we were limited in our analysis due to a lack of a non-septic matched cohort to further delineate the sensitivity and specificity of each model. Given the nature of burn sepsis diagnosis, where no single gold standard exists, examining diagnostic criteria would not be reliable without a true positive model for reference. Thus our study aimed to examine the predictive validity of these criteria against a reference of a prospective clinical diagnosis of sepsis instead, which we feel is an adequate substitution in lieu of a gold standard.

This study has various weaknesses and limitations. One limitation of this study is its inherent lack of reproducibility given the absence of a widely-accepted definition for the clinical diagnosis of burn sepsis, the very problem that definitions like ABA and Sepsis-3 attempt to solve. We therefore reduce reproducibility and objectivity with our suggestion to identify sepsis prospectively. Despite this, we believe that the clinical burn team is correct more often in evaluating for sepsis than the measures of any particular criteria. To prove this hypothesis, we used the clinical team’s definition of sepsis as the reference model. This is somewhat a weakness as the data feels incomplete; e.g. How many patients were positive for sepsis by the Sepsis-3 guidelines but negative by the team’s assessment? Why did these patients not get picked up and were they septic or not? What were the outcomes for the patients that were diagnosed as septic by the Sepsis-3 criteria, but not by the burn team?

In terms of future follow-up studies, organizing a system of prospectively documenting all relevant predictors for each criterion routinely, coupled with gathering prospective clinical diagnoses of sepsis when appropriate will provide data for a more detailed analysis of each sepsis criteria. In addition, this will grant us the opportunity to identify other predictors and novel combinations of predictors that might correlate better with the development of sepsis in burn patients and offer suggestions to improve upon the current existing criterion. Further studies in this matter will be a key component of augmenting provisions of care and patient outcomes. Currently, the lack of a standardized definition for sepsis for burn patients puts them at risk of suboptimal burn management and increased risk for complications. Furthermore, the lack of a coherent definition also poses barriers towards coordinating multi-center trials aimed studying burn sepsis.

The early identification and proper diagnosis of sepsis in the burn patient with prompt IV antibiotics is imperative to achieve a good outcome. In our study, we evaluated the performance of these three sepsis criteria in diagnosing the early phase of sepsis by applying them in the 48-hour period prior to the clinical diagnosis made by a physician. While a purely clinical diagnosis may lead to over-treatment of potentially septic-looking patients, it calls into discussion whether it is better medical practice to treat based on clinical suspicion of sepsis early on, or to delay treatment until the diagnosis is absolutely certain. We hypothesize and believe that the benefits of early antibiotic administration while sepsis is “developing” outweighs the risks in the setting of burn care, but certainly do acknowledge the need to balance this with antibiotic stewardship and the harm of potential adverse effects.

Previous investigations have shown the Sepsis-3 Consensus Definition to be a great initial screening tool for physicians in the general ICU setting.¹⁷ However, more recent studies have called into question the efficacy of the Sepsis-3 criteria in patients with more severe thermal injuries.¹⁸ Our study has also demonstrated its promise to identify early sepsis in burn trauma, while confirmation of the diagnosis will still require further clinical judgement and appropriate follow-up investigations. In the current state of burn care, burn-specialized clinicians must continue to consider the patient’s entire clinical status and progression when monitoring for sepsis. As such, the prospective clinical diagnosis of sepsis in the burn patient population should continue to remain the standard until adequate sepsis criteria are developed.

CONCLUSIONS

In summary, sepsis and other infections account for the majority of morbidity and mortality in burn patients. The new Sepsis-3 Consensus Definition developed in 2016, though not originally developed for burn patients specifically, outperformed the other burn-specific sepsis criteria at predicting the onset of sepsis. Early identification and initiation of treatment of sepsis remains a crucial aspect of proper burn care, and further development of diagnostic criteria for physicians to recognize early sepsis will be a crucial next step to improving outcomes for this population of patients. Current burn-specific sepsis criteria do not have the predictability to be reliably used in isolation in the clinical setting. A prospective clinical diagnosis made by an experienced physician analyzing the trends in a burn patient’s overall clinical picture will always be superior to a snapshot diagnosis using any single criterion. Therefore, we suggest a prospective clinical diagnosis for burn sepsis should remain the “standard care” until a proven and reliable sepsis criterion is developed for this population.