The EGS Grading Scale For Skin And Soft Tissue Infections Is Predictive Of Poor Outcomes : A Multicenter Validation Study.

Abstract

Introduction:

Over the last five years, the American Association for the Surgery of Trauma (AAST) has developed grading scales for Emergency General Surgery (EGS) diseases. In a prior validation study using diverticulitis, the grading scales were predictive of complications and length of stay. As EGS encompasses diverse diseases, the purpose of this study was to validate the grading scale concept against a different disease process with a higher associated mortality. We hypothesized that the grading scale would be predictive of complications, length of stay and mortality in skin and soft tissue infections (STI).

Methods:

This multi-institutional trial encompassed 12 centers. Data collected included demographic variables, disease characteristics and outcomes such as mortality, overall complications, hospital and ICU length of stay. The EGS scale for STI was used to grade each infection and two surgeons graded each case to evaluate inter-rater reliability.

Results:

1170 patients were included in this study. Inter-rater reliability was moderate (kappa coefficient 0.472-0.642, with 64-76% agreement). Higher grades (IV and V) corresponded to significantly higher LRINEC scores when compared with lower EGS grades. Patients with grade IV and V STI had significantly increased odds of all complications, as well as ICU and overall length of stay. These associations remained significant in logistic regression controlling for age, gender, comorbidities, mental status and hospital-level volume. Grade V disease was significantly associated with mortality as well.

Conclusion:

This validation effort demonstrates that Grade IV and V STI are significantly predictive of complications, hospital length of stay and mortality. Though predictive ability does not improve linearly with STI grade, this is consistent with the clinical disease process, in which lower grades represent cellulitis and abscess and higher grades are invasive infections. This second validation study confirms the EGS grading scale as predictive, and easily used, in disparate disease processes.

Study Type :

Prognostic/Epidemiologic retrospective multicenter trial

Level of Evidence :

III

Introduction

Over the last five years, the American Association for the Surgery of Trauma (AAST) has developed grading scales for the most prevalent emergency general surgery (EGS) disease processes. These grading scales were inspired by the original Organ Injury Severity scales developed by the AAST in the mid-1980’s.^1-3 Formulated from expert consensus opinion of AAST committee members and highly structured in format, the EGS scales rate the anatomic severity of disease processes from mild to severe, incorporating aspects of clinical presentation, radiographic findings as well as operative and pathologic data when available. The purpose of these scales was not only to prognosticate outcomes but to also create a standardized language used to categorize diseases as patients moved through an increasingly regionalized system of care. Additionally, the grading scales provide an important reference when constructing EGS data registries and conducting research. ⁴

The current EGS grading scales encompass twenty of the most common disease processes – including infectious, hemorrhagic and ulcerative diseases. Validation of these AAST EGS scales began by studying acute colonic diverticulitis, as it represented a common and well-understood disease process. In both an initial pilot study and a subsequent prospective, observational multicenter study, the grading scale demonstrated significant associations between higher-grade disease and outcomes such as the need for operative intervention, subsequent complications, length of stay, 30-day readmission rates and mortality. The studies also demonstrated good inter-rater reliability.^5-6

The disease processes addressed by the EGS grading scales represent quite disparate physiologic and pathologic processes however. The prognostic ability of the grading scale for acute colonic diverticulitis, though reassuring, does not necessarily translate to other diseases, such as acute bowel ischemia, breast infections or hernias. Therefore, the purpose of this study was to perform a follow-up multicenter validation study focusing on a different disease process with more profound associated physiologic derangement. Soft tissue infections (STI) were chosen due to the high incidence of operative interventions, significant physiologic insult to the patient and significant mortality associated with more severe grades of disease, as represented by necrotizing infections. Our hypothesis was that increasing disease grades would be associated with longer ICU and hospital length of stay, increased complication rates and increased mortality. We also hypothesized that inter-rater reliability would remain high despite applying a standardized scale to a different disease.

Methods

We conducted a multicenter retrospective cohort study as a project under the auspices of the AAST Patient Assessment Committee. The investigators at each participating center obtained local Institutional Review Board approval.

Study Design and Population

Most STIs involve cellulitis or abscess, rather than a necrotizing infection, and thus are concentrated in the low grades of the AAST grading scale. Therefore, a stratified sampling approach was used to enrich the study sample with uncommon, more severe cases. This allowed evaluation of the performance of the grading classification relatively equally across the spectrum of disease severity.

Investigators at each center submitted a de-identified list of all hospitalizations (an identification key with discharge diagnoses) from January 1, 2013 through August 31, 2015, with at least one International Classification of Diseases, 9^th Revision, Clinical Modification (ICD-9-CM) diagnosis code corresponding to an STI (Supplemental Digital Content). [For one Canadian center, we translated the relevant ICD-9-CM diagnosis codes to those in the International Classification of Diseases, 10^th Revision, Canada (ICD-10-CA).] One investigator categorized records involving an STI into seven strata (Supplemental Digital Content), then randomly selected records within each stratum to achieve targeted proportions of records per stratum and a total sample of 100 records/center. If some selected records were subsequently found to be ineligible, additional records were selected according to the same overall sampling scheme to achieve 100 eligible records/center (Figure 1).

Figure 1.

Flowchart demonstrate weighted patient sampling methods.

Investigators at each center reviewed selected records. Records were excluded if the patient was pregnant, had an advanced directive limiting life-sustaining care, underwent an operation for the index STI prior to the hospitalization of interest or, based on review of the full medical record, were found not to have had an STI.

Data Collection

From September 2016 to December 2017, investigators at each center collected information for eligible hospitalizations on demographic characteristics, diagnoses and procedures occurring during the hospitalization, components of the Sequential Organ Failure Assessment (SOFA) and the Laboratory Risk Indicator for Necrotizing Fasciitis (LRINEC) scores within the first 24 hours after presentation. Additional data collected included vasopressor requirements, AAST grade of STI (based separately on clinical, radiologic, operative, and pathologic criteria) (Figure 2), antibiotic therapies and complications that occurred during the index hospitalization (Appendix 1).⁴ Investigators entered data in a standard format via the AAST Multi-institutional Trials website.

Figure 2.

The EGS Soft Tissue Infection grading scale.

Exposure and Outcomes

The exposure, the AAST grade of STI, was defined as the maximum of the assignable grades determined from the clinical, radiologic, operative, or pathologic criteria. Two investigators at each center assigned grades independently. The primary grader entries were utilized in the main analysis.

Complications were the primary outcome in this study. Complications were compiled in two ways for this study. Each center was asked to record incidence of myocardial infarction, stroke, urinary tract infection and pneumonia as separate data points. Secondly, abstractors were also allowed to write-in pertinent complications under an ‘other complication’ variable. The ‘other complications’ were used to create a composite complication variable for the purposes of analysis. Disorders included in the composite variable are listed in Appendix 1. Additional outcomes included ICU and hospital length of stay, 30-day readmissions and mortality.

Analysis

Frequency distributions of categorical variables, mean (±standard deviation), and median (interquartile range) for normally and non-normally distributed continuous variables, were calculated. Associations between the STI grade and mortality, complications and 30-day readmission were assessed using Pearson’s chi-square test and univariate logistic regression. To examine associations between grade and hospital and ICU lengths of stay and LRINEC scores, ANOVA and Tukey-adjusted post-hoc pairwise comparisons of means were used.

Multivariable mixed-effect logistic regression was used to assess the relationship between STI grade and the composite complication outcome, readmission, death, and any complication. Similarly, Poisson regression was used to measure the association between grade and hospital and ICU lengths of stay. These models were adjusted for potential confounding factors selected a priori: age, gender, insurance status, admission Glasgow Coma Scale score (GCS), admission mean arterial pressure (MAP), total number of beds in the admitting hospital, hospital status (academic or non-teaching), urban or rural location, trauma center status and annual trauma center volume. Robust variance estimates were used to account for possible correlation of observations within centers.

Inter-rater reliability of grade assignment was assessed using the kappa coefficient and Cronbach’s alpha.⁷ Data were reported as the mean ± standard deviation except where specified otherwise and set the alpha at 0.05 for all tests. Results are reported for the logistic regression analyses as the adjusted odds ratios for each AAST grade, relative to Grade I, and those of the Poisson regression analyses as incidence rate ratios (IRR) relative to Grade I, together with their respective 95% confidence intervals (95% CI). 95% confidence intervals were computed for the kappa statistics using bootstrapping methods with 1000 replications. Stata SE version 14.2 was used for these analyses.

Results

12 centers contributed data for this study. 75% of centers were considered academic, compared to 25% non-teaching hospitals. 75% were located in urban centers, 8% were suburban, 8% were considered rural and 9% were other. Academic centers had a mean of 13 surgeons (± 5.6) and 878 beds (± 570). Non-teaching hospitals had a mean of 9 surgeons (± 2) on staff and 372 beds (± 218). There were no statistical differences between groups.

1170 patients were included in this study. The mean age was 53 years (± 18.7) and 62% of the patients were male. 54.5% of patients were Caucasian, 17% were African-American, 10.5% were Hispanic and 0.3% were of Asian descent. 16.8% of patients had private insurance, 57% had Medicare/Medicaid, 14.7% had no insurance and 11.6% had an other form of insurance(Table 1). Overall, 69.7% were seen by a surgeon during their admission and 60.4% required an operative intervention. Overall mortality was 1.7%

Table 1.

Patient characteristics stratified by STI grade

	Grade I (n=312)	Grade II (n=117)	Grade III (n=287)	Grade IV (n=181)	Grade V (n=243)	p-value
Age* (years)	52.1 (SD 22.7)	54.9 (SD 19)	49.9 (SD 17.5)	54.4 (SD 14.6)	55.2 (SD 15.8)	0.0014
Gender  % male	56.1%	59%	56.5%	68%	70.8%	0.0007
Race
Caucasian	55.8%	51.3%	54.7%	48.1%	56.8	0.0002
Afr. Amer.	13.8%	22.2%	14.6%	17.7%	21.4%
Hispanic	8.7%	6%	9.8%	11.6%	14.8%
Asian	0.6%	0.9%	0%	0%	0.4%
Other	21.2%	19.7%	20.9%	22.7%	6.6%
Insurance
None	14.1%	12%	12.9%	12.7%	20.7%	0.0058
Private	15.4%	17.1%	15.7%	15%	18.6%
Medicare	57.4%	63.3%	64.1%	60.8%	45.5%
Other	13.1%	7.7%	7.3%	11.6%	15.3%
LRINEC †  score	2 (1, 4)	3 (1, 4)	3 (1, 4)	4 (3, 6)	4 (2, 6)	<0.0001

^*mean (standard deviation)

^†median (interquartile range)

Patients were sub-classified based upon their EGS STI grade. 312 patients were included in Group 1, 117 in Group 2, 287 in Group 3, 181 in Group 4 and 243 in Group 5. In all groups, there were significantly more males, more Caucasians and more patients funded by Medicare/Medicaid (Table 1). Increasing severity of disease was also significantly associated with higher LRINEC scores, higher incidence of surgical consult and total number of operative cases, longer ICU and hospital length of stay, as well as a higher incidence of mortality (Tables 1&2).

Table 2.

Unadjusted outcomes by STI grade

	Grade I (n=312)	Grade II (n=117)	Grade III (n=287)	Grade IV (n=181)	Grade V (n=243)	p-value
Need for Pressors*	3 (1.0%)	5 (4.3%)	3 (1.1%)	32 (18.1%)	37 (15.6%)	<0.0001
30 day Readmit*	36 (n=71) (50.1%)	16 (n=28) (57.1%)	29 (n=61) (47.5%)	18 (n=49) (36.7%)	34 (n=71) (47.9%)	0.4602
Recurrent Infection* (STI site)	39 (12.5%)	12 (10.3%)	32 (11.2%)	20 (11.1%)	49 (20.2%)	0.0115
General comp. ‡*	12 (3.9%)	4 (3.4%)	7 (2.4%)	13 (7.2%)	18 (7.4%)	0.0315
Surgical Consult†	28.5%	66.7%	79.4%	96.7%	96.3%	<0.0001
Total No. Surgeries†	0 (0, 0)	0 (0, 1)	1 (1, 1)	2 (1, 3)	2 (1, 4)	<0.0001
ICU days†	0 (0, 0)	0 (0, 0)	0 (0, 0)	0 (0, 6)	0 (0, 4)	<0.0001
Inpatient days	3 (1, 6)	6 (3, 10.5)	4 (2, 8)	13 (7, 23)	14 (7, 23)	<0.0001
Discharge
Home	81%	79%	86%	58%	50%	<0.0001
SNF/rehab	17%	21%	13%	42%	48%
Other	2%	0%	0.7%	0%	2.4%
Mortality % died	0.64%	3.42%	0.35%	4.97%	11.93%	<0.0001

^‡General complications include myocardial infarction, pneumonia, stroke and urinary tract infection.

^*absolute number (total percent)

^†median (interquartile range)

Unadjusted outcomes for STI are demonstrated in Table 2. There was a significant association with increased need for vasopressors with increasing STI grade. Increasing STI grade was also significantly associated with recurrent infection at the initial STI site, increased general complications (including pneumonia, myocardial infarction, stroke and urinary tract infection), ICU and hospital length of stay, discharge to locations other than home and mortality. There was no significant association with 30 day readmission.

Multivariable analysis was used to assess outcomes as well. A ‘composite complication’ outcome was created and included both medical and surgical complications (see Appendix 1). Compared to STI Grade I patients, patients with Grade IV and V disease had a significantly increased incidence of all complications (NSTI Grade I referent; Grade II OR 1.38 (95% CI 0.68, 2.53); Grade III OR 0.789 (95% CI 0.44, 1.41); Grade IV OR 2.13 (95% CI 1.22, 3.71); Grade V OR 2.41 (95% CI 1.42, 4.08)). There was no association between STI grade and the need for readmission within 30 days however (Grade II OR 1.69 (95% CI 0.89, 3.17); Grade III OR 1.04 (95% CI 0.49, 2.20); Grade IV OR 0.65 (95% CI 0.31, 1.38); Grade V OR 1.51 (95% CI 0.86, 2.67)) (Table 3).

Table 3.

Adjusted Outcomes by EGS STI Subgroup

	Odds/Incidence Rate Ratio (95% CI)
Composite Complications
Grade I	-ref-
Grade II	1.38 (0.68, 2.53)
Grade III	0.79 (0.44, 1.41)
Grade IV	2.13 (1.22, 3.71)
Grade V	2.41 (1.42, 4.08)
Vasopressor Use
Grade I	-ref-
Grade II	2.72 (0.89, 8.29)
Grade III	1.38 (0.18, 10.7)
Grade IV	11.4 (2.82, 46.2)
Grade V	8.68 (2.41, 31.3
ICU Length of Stay
Grade I	-ref-
Grade II	2.78 (2.06, 3.75
Grade III	1.64 (1.23, 2.19)
Grade IV	7.98 (6.32, 10.1)
Grade V	9.78 (7.77, 12.3)
Hospital Length of Stay
Grade I	-ref-
Grade II	1.47 (1.34, 1.62)
Grade III	1.33 (1.23, 1.43)
Grade IV	2.89 (2.69, 3.10)
Grade V	3.40 (3.18, 3.63)
30-day Readmission
Grade I	-ref-
Grade II	1.69 (0.89, 3.17)
Grade III	1.04 (0.49, 2.20)
Grade IV	0.65 (0.31, 1.38)
Grade V	1.51 (0.86, 2.67)
Mortality
Grade I	-ref-
Grade II	5.34 (1.21, 23.5)
Grade III	0.66 (0.07, 6.38)
Grade IV	2.65 (0.61, 11.4)
Grade V	15.2 (3.24, 71.1)

Infusion of vasopressor agents to support blood pressure in affected patients was significantly associated with higher grade STI’s (Grade II OR 2.72 (95% CI 0.89, 8.29); Grade III OR 1.38 (95% CI 0.18, 10.68); Grade IV OR 11.4 (95% CI 2.82, 46.16); Grade V OR 8.68 (95% CI 2.41, 31.30)). Increasing STI grade was also directly related to both intensive care unit length of stay (Grade II OR 2.78 (95% CI 2.06, 3.75); Grade III OR 1.64 (95% CI 1.23, 2.19); Grade IV OR 7.98 (95% CI 6.32, 10.09); Grade V OR 9.78 (95% CI 7.77, 12.31) and overall hospital length of stay (Grade II OR 1.47 (95% CI 1.34, 1.62); Grade III OR 1.33 (95% CI 1.23, 1.43); Grade IV OR 2.89 (95% CI 2.69, 3.10); Grade V OR 3.4 (95% CI 3.18, 3.63)). There was a bimodal peak in the risk of mortality, with a significant increase in mortality risk seen in patients with STI grades II and V disease (Grade II OR 5.34 (95% CI 1.21, 23.50); Grade III OR 0.66 (95% CI 0.07, 6.38); Grade IV OR 2.65 (95% CI 0.61, 11.43); Grade V OR 15.18 (95% CI 3.24, 71.12)) (Table 3).

Overall inter-rater reliability demonstrated a κ score of 0.668 (95% CI 0.624, 0.709, 74.2% concordance rate). Inter-rater reliability was also assessed for each of the four components of the STI score. Substantial inter-rater reliability was seen with grading of the operative component (κ score 0.642 (95% CI 0.571, 0.712), 76% concordance rate). The remaining three categories demonstrated moderate inter-rater reliability (clinical κ score 0.579 (95% CI 0.529, 0.625), 69% concordance; radiologic κ score 0.577 (95% CI 0.495, 0.661), 68% concordance; pathologic κ score 0.472 (95% CI 0.374, 0.568), 64% concordance).

Discussion

Since the inception of the EGS grading scale project, scales for over 20 conditions have been defined and published.^{4, 8} Validation of the grading scales has been underway since the earliest scales were available. The first validation study, a multicenter study, demonstrated good correlation between increasing severity of disease and patient outcomes.^5-6 Subsequent single institution studies have evaluated the validity of the EGS grading scales for acute appendicitis, acute cholecystitis and acute pancreatitis. In each of these studies, the EGS grading scale was found to be associated with outcomes. ^9-11

Though the EGS grading scale has been validated for a variety of different diseases, this study fills an important niche. In general, the EGS grading scales are specifically designed to address two main gaps. On the one hand, care of the complex EGS patient is becoming increasingly fragmented. As patients are transferred between, and handed-off within, institutions, a common language to communicate the anatomic severity of disease is helpful. Additionally, many centers are beginning to construct EGS data registries. A valid, reliable anatomic scoring system is necessary to classify severity of disease, allowing for improved benchmarking of quality of care and more accurate research on these diseases.

The current study focuses specifically on STIs. Necrotizing STIs (NSTIs) are uncommon, occurring in approximately 0.4/100,000 person-years and many surgeons may not see a true NSTI in their career.^12-15 Validating the accuracy of the grading scale for a disease process such as this is therefore especially important. Further, NSTIs have a high associated mortality, allowing validation of the grading scale’s ability to predict mortality across the spectrum of disease severity. Most other studies evaluating the validity of the AAST EGS grading scales have not had sufficient power to identify associations between grade and mortality, partly as many of the diseases studied to date have had a low mortality rate.^{14, 16}

The multicenter aspect of this study—with 1170 patients from 12 centers—allowed validation of the scale across a variety of institutions, practitioners, and patients. Because NSTIs (Grade IV and V disease) are rare compared to cellulitis and subcutaneous infections (Grade I and II disease) and abscess (Grade III disease), truly random sampling would result in a preponderance of low severity cases and little representation of more advanced STIs. To address this issue, we purposefully weighted our sampling scheme to allow a more equal distribution of patients from all five grades. This design allowed us to evaluate the association of grades with outcomes across the spectrum of STI severity.

The STI grading scale was also compared to the existing predictive tool for STI, the LRINEC score.^17-19 The LRINEC score incorporates physiologic data present at admission including white blood cell count, hemoglobin, serum glucose, serum sodium, serum creatinine and C-reactive protein. A score of greater than or equal to 6 is associated with a high risk of necrotizing infection. LRINEC scores were also calculated for all patients included in the study. The median LRINEC score for the most severe STI’s in this study, a median of 4 for grade 4 and 5 disease, was notably lower than the usual predictive threshold of 6 for LRINEC. Some debate exists in the literature regarding the sensitivity of the LRINEC score, though it remains the most widely used predictive model.^20-23 Increasing AAST EGS grades were associated with increasing LRINEC scores, despite the difference in predictive threshold.^20-23 Further refinement of the EGS grading scales may ultimately result in greater alignment with existing predictive models or potentially prove superior.

Specific data regarding patient-related complications were recorded (Appendix 1). Due to the large variety of complications, we used a composite ‘complication’ outcome. High-grade STIs (grade IV and V) were associated with the composite complications. We also assessed the association between STI grade and the need for vasopressor support. Not surprisingly, a significant association was observed for vasopressor support. By inference, the STI grading scale may thus be associated with hemodynamic instability and the physiologic status of the patient. Finally, STI grades were also associated with increased ICU and hospital lengths of stay.

The association between STI grade and mortality was not entirely linear. In general, despite the severity of the disease process, mortality was low at 1.7%. There were two peaks seen for mortality. The 15-fold increase in mortality related to Grade V disease likely represents an accurate association between disease severity and risk of death. However, when translating the STI scales, Grade I and II disease represent mild to severe cellulitis. The 5-fold increased risk of mortality associated with Grade II STIs (relative to Grade I) may be attributable to confounding comorbid conditions or it may reflect that even superficial necrotizing infections truly do increase mortality.

An additional purpose of this validation study was to evaluate the reliability of grading between different graders. The grading scale includes criteria for clinical, radiologic, operative and pathologic findings from which we derived a maximum score. There was substantial reliability when determining an overall grade. We observed the greatest inter-rater agreement with the operative score but less with the other criteria. These findings are similar to those observed in the previous multicenter validation study and indicate that the grades appear to be fairly reliable across providers, at disparate centers.⁶

After validating, the grading scales across a variety of disease processes in both single and multicenter formats, the EGS grading scales in general appear to be predictive of patient outcomes, are easy to use and are reliable between graders. In the future, early adopters of these scales for patient care and research should educate the larger community regarding the utility of the grading scales and promulgate their use. In particular, the importance of these data for registries cannot be overstated, as they will form the basis for a multitude of studies, especially in relation to relatively rare disease processes like NSTI. A further goal for the grading scales may be ongoing refinements, as needed, for particular disease processes. Such a refinement is underway in regards to grading acute cholecystitis with incorporation of a real-time, intra-operative component.¹¹ Specifically in regards to STI, inclusion of physiologic parameters at the time of admission, perhaps in the form of the LRINEC score, as well as comorbid data will likely strengthen this scale.

One important limitation of the EGS grading scale is that some information pertinent to assessing the grade, such as operative findings or pathology, are available only after treatment of the patient, largely limiting the grading to a retrospective role. Greater emphasis should be placed on determining grades early in the patient’s course (e.g., based only on initial physical exam and radiographic criteria), so that they have greater utility in influencing management decisions. Further, the grading scales do not incorporate any physiologic data or information on patient comorbidities. This limitation was a choice consciously made by the grading scale authors, to keep the grading scales easy to remember and use. These characteristics clearly impact patient outcomes and must be collected separately for research purposes, however. These data only include patients admitted to the participating hospitals. Patients with mild forms of cellulitis, or small abscesses treated in clinics or the emergency room, would not be identified and are excluded from this analysis. Finally, EGS diseases run the spectrum from infectious to gastrointestinal to thoracic and beyond. One common scoring system, though easy to remember, may result in loss of specificity and gaps in data, which may be important for individual diseases. This issue must be addressed via a robust registry or further refinement of individual grading scales.

Conclusion

This multicenter study demonstrates the validity of the AAST STI grading scale. Higher disease grades were significantly associated with a number of important patient outcomes. Additionally, inter-rater reliability of the grading was moderate to substantial. Use of this grading scale should be encouraged among those who care for STI patients, to improve communication between providers and allow better comparison of groups of patients with STIs for both research and quality measurement purposes. Future work will continue to refine some grading scales to better reflect specific characteristic of diseases and to address discrepancies in inter-rater reliability in use of the grading scale.