Risk Models of Operative Morbidities in 16,930 Critically Ill Surgical Patients Based on a Japanese Nationwide Database

Supplemental Digital Content is available in the text

Abstract

The aim of the study was to evaluate preoperative variables predictive of lethal morbidities in critically ill surgical patients at a national level.

There is no report of risk stratification for morbidities associated with mortality in critically ill patients with acute diffuse peritonitis (ADP).

We examined data from 16,930 patients operated during 2011 and 2012 in 1546 different hospitals for ADP identified in the National Clinical Database of Japan. We analyzed morbidities significantly associated with operative mortality. Based on 80% of the population, we calculated independent predictors for these morbidities. The risk factors were validated using the remaining 20%.

The operative mortality was 14.1%. Morbidity of any grade occurred in 40.2% of patients. Morbidities correlated with mortality, including septic shock, progressive renal insufficiency, prolonged ventilation >48 hours, systemic sepsis, central nervous system (CNS) morbidities, acute renal failure and pneumonia, and surgical site infection (SSI), were selected for risk models. A total of 18 to 29 preoperative variables were selected per morbidity and yielded excellent C-indices for each (septic shock: 0.851; progressive renal insufficiency: 0.878; prolonged ventilation >48 h: 0.849; systemic sepsis: 0.839; CNS morbidities: 0.848; acute renal failure: 0.868; pneumonia: 0.830; and SSI: 0.688).

We report the first risk stratification study on lethal morbidities in critically ill patients with ADP using a nationwide surgical database. These risk models will contribute to patient counseling and help predict which patients require more aggressive surgical and novel pharmacological interventions.

INTRODUCTION

Acute diffuse peritonitis (ADP) is defined as the uncontained spread of intraabdominal infection, rapidly proceeding beyond the source of infection into multiple (2–4) quadrants of the intraabdominal cavity.¹ Most patients diagnosed with ADP are critically ill and therefore require emergency surgery, regardless of the source of infection.^2–4 A high incidence of severe postoperative complications such as septic shock, pneumonia, and organ failure has resulted in a high mortality rate of approximately 30%, even in modern case series.⁴ Therefore, the identification of postoperative complications associated with mortality and their optimal treatment is necessary to improve outcomes. There have been risk models for mortality in critically ill patients. The Acute Physiology and Chronic Health Evaluation II (APACHE II) score,⁵ Sequential Organ Failure Assessment score,⁶ and Mannheim Peritonitis Index⁷ have all been shown to be quite effective for predicting mortality in critically ill patients. However, there has been no risk model for the morbidity of critically ill patients using a nationwide database.

American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) risk models are powerful predictors of specific morbidities and mortality associated with gastrointestinal surgery.^8–10 However, there has been no nationwide analysis of critically ill surgical patients. In one regional report, Turner et al¹¹ showed that ACS-NSQIP criteria were associated with high APACHE II scores and poor outcomes in 340 surgical patients (mortality: 20.6%) treated in the intensive care unit of the University of Maryland Medical Center (Baltimore, MD). They found that APACHE II score predictions were consistent with ACS-NSQIP postoperative outcomes. This observation prompted us to hypothesize that ACS-NSQIP preoperative variables could be used to predict both postoperative morbidities and mortalities in ADP patients.

The National Clinical Database (NCD) in Japan, which commenced patient registration in January 2011, is a nationwide project linked to the surgical board certification system. ^12,13 Submitting cases to the NCD is a prerequisite for all member institutions of both the Japan Surgical Society and Japanese Society of Gastroenterological Surgery, and only registered cases can be used for board certification. The NCD collaborates with the ACS-NSQIP¹⁰: they share the common goal of developing a standardized surgery database to achieve an improvement in treatment quality.¹⁴

Previously, we reported that patients with ADP are critically ill, most require emergency surgery, and their 30-day mortality and 90-day in-hospital mortality rates are 9% and 13.9%, respectively.¹⁵ In this study, we used data from 16,930 patients with ADP treated in 2011 and 2012 and registered with the NCD to create risk models for postoperative morbidities associated with mortality.

METHODS

Patient Selection

The NCD is a nationwide project associated with the board certification system of surgery in Japan into which data from over 1,200,000 surgical cases treated at over 3500 hospitals are entered annually. We have created risk models of mortality for the 8 surgical procedures (esophagectomy, total gastrectomy, distal gastrectomy, right hemicolectomy, low anterior resection, hepatectomy, pancreaticoduodenectomy, and ADP) using NCD data sets, and the respective model was published separately,^15–22 and the results were summarized as a review article.¹³ Thus, patient selection, preoperative and perioperative variables, and ethics consideration were quite consistent between the studies. The NCD continuously recruits individuals who approve these data, members of various departments in charge of cases, and data entry officers through a web-based data management system; thus, the traceability of the data is assured.¹² In addition, the project constantly validates the consistency of these data by the inspection of randomly chosen institutions. Current laws, ordinances, and guidelines regarding the confidentiality of data are observed. Patients agree for their data to be included in research projects by using presumed consent with opt-out through the Web page and/or a notice of each hospital.²⁰ The NCD project was approved on November 2010 by Japan Surgical Society Ethics Committee.

In this study, we focused on ADP in the Gastrointestinal Surgery section of the NCD. In the NCD, we identified 16,930 patients who underwent surgery for ADP in 2011 to 2012. Patients who declined to have their records entered in the NCD were excluded from our analysis. Records with missing data on patient age, sex, or status, 30 days after surgery were also excluded.

Preoperative and Perioperative Variables

The preoperative and perioperative variables used by the NCD are almost identical to those used by the ACS-NSQIP (http://site.acsnsqip.org/wp-content/uploads/2013/10/ACSNSQIP.PUF_UserGuide.2012.pdf#search=’user+guide+for+the+2012+ACS+NSQIP). All variables, definitions, and inclusion criteria regarding the NCD are accessible to participating institutions on its website (http://www.ncd.or.jp/), which also features an E-learning system to instruct participants in how to input consistent data. The potential independent variables were previously described.^13,15–22 These included patient demographics, preexisting comorbidities, preoperative laboratory values, and perioperative data (Table 1 ).

TABLE 1.

Preoperative Risk Profiles and Laboratory Data of the Study Population

Outcome Measures (Mortality and Postoperative Occurrences)

We calculated the 30-day mortality and operative mortality. The former was defined as death within 30 days of surgery, regardless of the patient's geographical location, even if the patient had been discharged from the hospital. The latter was defined as death within the index hospitalization period, regardless of the length of hospital stay (up to 90 days), as well as any death after discharge within 30 days of surgery.

The postoperative morbidities that occurred within 30 days of surgery included relaparotomy within 30 days of surgery; wound-related morbidities (superficial incisional surgical site infection [SSI], deep incisional SSI, organ/space SSI, wound disruption); respiratory morbidities (pneumonia, unplanned intubation, pulmonary embolism, ventilation >48 hours); urinary tract morbidities (progressive renal insufficiency, acute renal failure, urinary tract infection); central nervous system (CNS) morbidities (stroke/cerebrovascular accident [CVA], coma for <24 hours, peripheral nerve injury); cardiac morbidities (cardiac arrest, myocardial infarction); and other occurrences (bleeding 1–4 u or ≥5 u red blood cells, deep-vein thrombosis/thrombophlebitis, septic shock, severe sepsis, systemic inflammatory response syndrome [SIRS]).

Statistical Analysis

We used IBM SPSS Statistics for Windows (Version 20; IBM Corp, Armonk, NY) for data analysis. Univariate analysis of the data was performed using Fisher exact test, the unpaired Student t test, and the Mann–Whitney U test. Correlations between each morbidity and operative mortality and between respective morbidities were analyzed using the Pearson product–moment correlation.

Data were randomly assigned into2 subsets that were split 80/20: the first for model development and the second for validation. The 8 sets of logistic models (septic shock, systemic sepsis, progressive renal insufficiency, acute renal failure, ventilation >48 hours, pneumonia, CNS morbidities, and SSI) were constructed for dataset development using step-wise selection of the predictors with a probability (P) value for inclusion of 0.05. A “goodness-of-fit” test was performed to assess how well the model discriminated between patients with or without respective morbidities. Receiver operating characteristic (ROC) curves for respective morbidities were created for the validation dataset. A ROC curve is a plot of a test's true-positive rate (sensitivity) versus its false-positive rate (1−specificity).

RESULTS

Preoperative Risk Profiles and Laboratory Data of the Study Population

The demographic data and risk profile of 16,930 patients with ADP are shown in Table 1 . The patient population had a mean age of 64.9 ± 18.6 years (range: 0–106 years), and 60.5% (n = 10,248) were male. In this population, 37.7% arrived at hospital by ambulance, and 92.9% required emergency surgery. Their original disease and associated operative mortalities were acute peritonitis (15.1%), appendicitis (1%), gastroduodenal ulcer/perforation (9.5%), intestinal perforation (18.4%), intestinal obstruction (18.9%), cholecystitis/cholangitis (13.3%), and vascular insufficiency (31.2%). These proportions and mortalities are consistent with findings from 2011.¹⁵

An abbreviated risk profile for the study population is also shown in Table 1 . In brief, 58.4% of the patient population had an American Society of Anesthesiologists (ASA) classification of III–V, partial/total dependency for activities of daily living (ADL) was 41.2%, 0.5% of patients had body mass index (BMI) of >30 kg/m², and 5.1% of patients had a weight loss of >10%. With regard to preexisting comorbidities, failure of various organs occurred in a percentage of patients, including ventilator dependence (3.8%), congestive heart failure (2.6%), and acute renal failure (4.4%). Signs of systemic sepsis were evident in 30.9% of patients. Blood transfusion was required in 4.1% of patients. An ASA classification of >IV and V and organ failure were associated with an operative mortality rate of >40%.

Postoperative Occurrences in Patients with ADP

The 30-day mortality and operative mortality rates after surgery for ADP were 8.8% (1482) and 14.1% (2385), respectively. The incidences of various morbidities and percentage of consequent patient deaths are shown in Table 2. The postoperative morbidities that led to a high percentage of deaths (>40%) included transfusion (1–4 U: 43.5%; >5 U: 52.2%), prolonged ventilation (45.6%), unplanned intubation (51.4%), pneumonia (43%), cardiac and CNS morbidities (90.3% and 64.8%, respectively), acute renal failure (57.1%), progressive renal insufficiency (55.6%), any systemic sepsis (41%), and septic shock (55.8%). These morbidities occurred at a relatively high incidence (4.8%–15%) excepting cardiac morbidities (2.5%). SSI of any type, including organ space, deep incisional, and superficial incisional, occurred in 23.2% of patients and led to an operative mortality rate of 20.8%.

TABLE 1 (Continued).

Preoperative Risk Profiles and Laboratory Data of the Study Population

Correlation Between Postoperative Morbidities and Operative Mortality

Correlation between 30-day operative mortality rates and postoperative morbidities were analyzed using the Pearson product–moment correlation. The morbidities highly correlated with mortality (top 7) as well as SSI as the most representative complication of ADP were selected and are compared in Table 3. A better correlation with postoperative morbidities was found when operative rather than 30-day mortality was used. Among the postoperative morbidities, septic shock, progressive renal insufficiency, and ventilation >48 hours were highly correlated with each other (r >0.5). In contrast, SSI was only moderately correlated with systemic sepsis, and weakly correlated with ventilation >48 hours.

TABLE 2.

Postoperative Occurrences After ADP Surgery

Model Results and Performance

We developed risk models for postoperative morbidities with a relatively high incidence associated with high mortality (Table 4 ; Supplemental Table, http://links.lww.com/MD/A344, with 95% confidence intervals [CIs]). The postoperative morbidities selected correlated well with operative mortality. Septic shock, systemic sepsis (SIRS, sepsis, or septic shock), progressive renal insufficiency, acute renal failure, ventilation >48 hours, pneumonia, and CNS morbidities were selected, and SSI was also included as the most frequent morbidity.

TABLE 3.

Correlation Between Operative Mortality and Respective Postoperative Occurrences

TABLE 4.

Risk Models of Postoperative Occurrences After ADP Surgery

TABLE 4 (Continued).

Risk Models of Postoperative Occurrences After ADP Surgery

The logistic models of these morbidities with odds ratios are shown in Table 4 . The morbidities with a 95% CI showing statistical significance are shown in the Supplemental Table, http://links.lww.com/MD/A344. To evaluate the performance of the models, the C-index (a measure of model discrimination), which was the area under the ROC curve, was calculated for the validation sets (Figure 1). The C-indices and 95% CIs of each occurrence were 0.851 (0.841–0.860) for septic shock, 0.878 (0.870–0.887) for progressive renal insufficiency, 0.849 (0.841–0.858) for ventilation >48 hours, 0.848 (0.835–0.862) for CNS morbidities, 0.868 (0.856–0.880) for acute renal failure, 0.830 (0.819–0.840) for pneumonia, and 0.851 (0.841–0.860) for systemic sepsis. The C-index of SSI showed a weaker correlation (0.688 [0.677–0.698]) than other morbidities.

FIGURE 1.

Receiver operating characteristic (ROC) curves of each postoperative complication was shown with the C-indices and 95% CIs of each occurrence. ROC = receiver operating characteristic, CIs = confidence intervals.

A total of 18 to 29 preoperative variables were selected as risk factors of each complication. Age, ASA classification, preoperative ventilation or pneumonia, acute renal failure, blood transfusion, and systemic sepsis, as well as selected preoperative laboratory values suggestive of severe infection and organ failure, were captured in the risk models as predictors of most of the complications.

DISCUSSION

We hypothesized that ACS-NSQIP preoperative variables could be used to predict both postoperative morbidities and mortalities in ADP patients. In total, 93% of 16,930 patients with ADP included in this study required emergency surgery, and the overall operative mortality was 14.1%. This was comparable with the findings of a previous analysis using NCD data from 2011,¹⁵ in which 93.1% of patients with ADP required emergency surgery, and the overall operative mortality was 8.8%. This suggests that there is a consistent population of critically ill surgical patients who require emergency surgery in Japan. By examining the data of a large number of patients with ADP, we were able to identify the postoperative complications associated with mortality and create risk models for each complication. Septic shock, progressive renal insufficiency, ventilation >48 hours and systemic sepsis were moderately correlated (r >0.36) with operative mortality, whereas CNS morbidities, acute renal failure, and pneumonia were weakly (0.2 < r ≤ 0.35) correlated with operative mortality. For these complications, risk models showed excellent C-indices (>0.830) in the validation dataset. To our knowledge, this is the first report to successfully show and validate using a large-scale dataset that the preoperative variables of the ACS-NSQIP can predict postoperative morbidities in critical ill patients.

The prediction of postoperative complications is essential to the decision-making process before surgery, and useful to identify patients eligible for participation in the evaluation of novel pharmacologic interventions^23,24 or more aggressive surgical interventions. In the past, several scoring systems have been used to predict complications.^25–31 ASA score is a useful predictor for mortality,^25,26 but suffers from its reproducibility because of subjective parameters.²⁶ APACHE II was developed in a mixed group of medical and surgical patients.²⁷ It failed to predict the development of multiple organ failure syndrome or mortality with clinical utility in postoperative surgical patients.²⁸ Physiological and Operative Severity Score for the enUmeration of Mortality and Morbidity has been studied as a possible surgical audit system²⁹; however, it seems to overestimate mortality, particularly for the low risk group.^30,31 A reliable model for predicting complications can only be based on the accurately recorded incidences of those complications. A comparison of the outcomes of patients with ADP registered with the NCD in 2011 with those registered in 2012 revealed that mortality and morbidities were highly correlated between these years (r = 0.9932; Supplemental Figure, http://links.lww.com/MD/A344). The thorough data retrieval system of the NCD and clinically clear entity of ADP made it possible to create successful risk models for these morbidities.

Severe sepsis/septic shock, defined as the presence of acute organ dysfunction in the context of infection, has a mortality rate of approximately 25% to 35%,^32,33 but which can exceed 70%.^34,35 Anaya and Nathens³⁶ analyzed risk factors of severe sepsis in 11,202 patients using Washington State administrative hospital discharge data. They identified 11% with severe sepsis, which was present in 424 (62%) of the 686 decedents, and showed that source of infection, extent of peritonitis, increasing age, and preexisting organ dysfunction were independently associated with severe sepsis. Our findings on the mortality of patients with ADP were consistent with their study. The mortality of patients with ADP as a result of appendicitis was low (1%) compared with that associated with other causes such as intestinal/gastroduodenal perforation (18.4%/9.5%), vascular insufficiency (31.2%), and cholecystitis/cholangitis (13.3%). Regarding peritonitis, when it is localized within an abscess, the operative mortality rate of cases registered with the NCD was relatively low (4.6%; 254 deaths/5470 cases) compared with that of patients with ADP (14.1%). This study provides more reliable information on clinical variables and laboratory data compared with the findings of Anaya and Nathens.³⁶ We were able to select significant variables to predict each complication, and discrimination and calibration using validation tests clearly showed the excellent performance of these models.

It is interesting to note that the risk models for morbidities moderately associated with mortality (septic shock, any systemic sepsis, renal failure, acute renal failure, prolonged ventilation, pneumonia, and CNS morbidities) picked up similar variables as risk factors—age, ADL status, ASA classification, blood transfusions, and systemic sepsis—to those found to be risk factors of mortality in patients with ADP.¹⁵ Preoperative variables associated with organ dysfunction tended to be included as risk factors in most of the risk models: preoperative ventilation/pneumonia, acute renal failure, bleeding disorders, low white blood cell count, low albumin level, and elevation of blood urea nitrogen.¹⁵ High serum sodium levels, indicative of severe dehydration in patients, were also identified. In contrast, the risk model for SSI, which was poorly associated with mortality (r = 0.107), showed a relatively low C-index (0.688) compared with the other risk models. Risk factors such as pulmonary, renal, and cerebral disorders were not included in the risk model. The key part of these risk models is that variables that were not included as risk factors of mortality were picked up as predictors of morbidities leading to mortality. This will help to improve the postoperative management of patients with ADP.

There are several limitations to this study. First, although these risk models for morbidities effectively predicted their occurrence based on preoperative variables, the source of infection and degree of its control would affect mortality and morbidity. These intraoperative parameters will be evaluated in a future study. Second, in the NCD data-entry system, the final outcome of each morbidity, whether it improved, was unresolved, led to death, and was not recorded. It is not possible to relate each morbidity directly to mortality, although most fatal cases feature multiple organ failure at the end.

ADP is a clinically distinct entity requiring life-saving emergency surgery and intensive care. We created risk models for morbidities in critically ill patients with ADP, using variables recorded by the NCD comparable to those of the ACS-NSQIP, and these models performed well. These models could be formatted to feed information back to the NCD and can be expected to improve the quality of the surgical and postoperative care of patients with ADP.