Whipple-Specific Complications Result in Prolonged Length of Stay Not Accounted for in ACS-NSQIP Surgical Risk Calculator

Abstract

Background

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) Surgical Risk Calculator was developed to help counsel patients regarding estimated postoperative risk for a variety of surgical complications. This retrospective single institutional study examined the calculator’s ability to accurately predict complications and length of hospital stay (LOS) in patients who had undergone a Pancreaticoduodenectomy (PD) at our institution.

Methods

165 patients at Washington University School of Medicine who underwent a PD from 8/2011 to 7/2013 were included. Surgical complication risk as determined by the ACS-NSQIP Surgical Risk Calculator were compared to actual 30 day complications. PD complications not accounted for by the calculator were compared to those without PD-specific complications.

Results

Overall predicted LOS was significantly shorter than actual duration of hospitalization (median 8.5 vs 8.0 days; p <0.001). 38% patients (n=62) with Whipple-specific complication demonstrated a significant increase in LOS (8.0 vs. 12.2 days; p<0.0001).

Discussion

A large proportion of complications experienced after PD are pancreas-specific, accounting for the difference in predicted vs actual LOS and providing rationale for future development of PD specific risk models.

INTRODUCTION

The Whipple Procedure (Pancreaticoduodenectomy, PD) is a complex surgical operation often performed to treat suspected as well as confirmed malignancies in the head of the pancreas. Until the 1970s the perioperative mortality and morbidity of the Whipple procedure was over 20% and over 50%, respectively.^(1–3) Over time, incremental advances have been made, and while morbidity remains high, mortality, in most centers, has dropped considerably and is consistently below 4%.^(4–8) Standardized reporting of complications, using American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP), has confirmed the high morbidity and relatively stable death rate of 2–3%. For example, a 2011 study by Greenblatt et. al. reported that 28.1% of patients who underwent a PD suffered from a serious complication and 2.6% died within 30 days.⁽⁹⁾

With the new health care overhaul and the emphasis on both quality and cost effective care, the dollar value tied to complications and resultant subsequent readmissions is increasingly emphasized.^{(10, 11)} Each Surgical Site Infection (SSI) was calculated to add $10,497 of cost and 4.3 additional days of hospital stay.⁽¹²⁾ Thus, preventing complications improves patient care and saves health care dollars.⁽¹³⁾ In order to provide informed consent, patients must be counseled accurately on perioperative risks and the likelihood of complications. Additionally, it allows patients the opportunity to modify risk factors that make them high risk prior to surgery, or perhaps more importantly it provides an opportunity for patients to opt out of surgery when the perceived risks outweigh the potential benefits. As mentioned, there is a high morbidity of nearly 30% for all comers after PD (9), and given the dismal overall prognosis of pancreatic adenocarcinoma, with a 5 year survival of 5%⁽¹⁴⁾ and only 20–27% after resection, more conservative management options may provide a better alternative for frail patients.^{(15, 16)}

The ACS NSQIP Surgical Risk Calculator⁽¹⁷⁾ was developed as a tool to help counsel patients regarding their 30-day postoperative risk for a variety of major complications. In an easily-accessible online tool, a surgeon or patient may input 21 different preoperative variables plus the CPT code and receive an estimated risk percentage for 9 different outcomes, 2 general groups (serious complication and any complication), as well as a predicted length of stay. The estimated risk is also compared to the average of all patients who have undergone that specific procedure.⁽¹⁸⁾ The calculator was validated using 1.4 million patients and over 2,500 CPT codes. Physicians are being encouraged to use this calculator to assist in the consent process. There is sparse data to support its effectiveness as a teaching aide and less to suggest it will alter the patient’s decision during an informed consent process. A recent publication looking at laparoscopic colectomy demonstrates that there is growing interest in looking at the effectiveness of using NSQIP Surgical Risk Calculator as a risk estimator, at the single-institution level.⁽¹⁹⁾

Our study aims to determine the calculator’s ability to accurately predict risk of complications and length of hospital stay (LOS) in a group of patients who have undergone a PD at a single academic medical center over a two-year time span. Additionally, we sought to determine the important complications not captured by the ACS NSQIP Surgical Risk Calculator. We hypothesized that due to its validation in a large population of patients, the calculator correctly estimates the percent chance of complications after PD and will appropriately stratify high risk and low risk patients for morbidity and mortality after PD.

METHODS

Data consisted of a total of 165 patients who underwent an elective Whipple procedure at Barnes-Jewish Hospital (BJH) from August 2011 to July 2013. IRB and HRPO approval was given, and a prospective pancreatic surgery complications database was used to acquire our patient population. Using patient identifiers, the predictive data required for entry into the ACS NSQIP Surgical Risk Calculator was gathered from the database, and the patients’ electronic medical record. Patient demographic data entered into the calculator included the CPT code of the procedure, age, sex, patient functional status, whether the case was performed as an emergency, ASA class, wound class (clean/contaminated for every patient in Whipple procedure), steroid use for chronic condition, ascites within 30 days prior to surgery, systemic sepsis within 48 hours prior to surgery, ventilator dependency, disseminated cancer, diabetes (insulin-dependent, oral medications or none), previous cardiac event, congestive heart failure (CHF) within 30 days prior to surgery, dyspnea (at rest, with moderate exertion or none), current smoker within one year, history of severe chronic obstructive pulmonary disease (COPD), dialysis, acute renal failure (ARF), and height and weight for BMI calculation.

Data for each patient were entered into the ACS NSQIP calculator individually, and the results of the calculator were recorded to the nearest tenth of a percentage. The “other potential treatment options” tab on the ACS NSQIP calculator was set at “none”, and the “Surgeon Adjustment of Risks” was set at “1 - No adjustment necessary” for all patients. The standard categories of predicted outcome data (in estimated risk percentage) were the output of the risk calculator. These categories included nine individual outcomes and two overall groups. Those outcomes that were predicted individually were: pneumonia (PNA), cardiac complication (CA/MI; including cardiac arrest and myocardial infarction), surgical site infection (SSI; including superficial incisional, deep incisional, organ space SSI), urinary tract infection (UTI), venous thromboembolism/blood clot (VTE), acute renal failure (ARF; including either progressive renal insufficiency or requiring dialysis), return to the operating room for additional surgery that was not planned at the time of the initial surgery, death, and predicted length of hospital stay (LOS). In addition to the individual categories, the estimated percentage chance of risk of two groups of multiple complications were also given: serious complication (CA/MI, PNA, progressive renal insufficiency, ARF, pulmonary embolism (PE), diverticulitis, return to the operating room, deep incisional SSI, organ space SSI, systemic sepsis, unplanned intubation, UTI, and wound disruption) and any complication (superficial incisional SSI, deep incisional SSI, organ space SSI, wound disruption, PNA, unplanned intubation, PE, diverticulitis, ventilator > 48 hours, progressive renal insufficiency, ARF, UTI, stroke, CA/MI return to the operating room, and systemic sepsis). The categorical (yes/no) actual outcomes data and percentage incidence from the prospective Whipple complications database were compared to the continuous output of the ACS NSQIP surgical risk calculator using regression analysis to determine statistical significance. A second test, a Kruskal-Wallis test, was used to assess predictive ability of the calculator. It did so because as a retrospective study, those who experienced complications in any given category of the calculator were separated from those who did not experience those complications, and the means of those two groups were compared to see if the output of the no’s was remarkably different from yesses. Continuous estimated and actual length of stay data was compared for statistical significance using a paired t-test.

Whipple specific complications (delayed gastric emptying⁽²⁰⁾, pancreatic leak/leak⁽²¹⁾, bleeding or ileus) that were unaccounted for in the ACS risk calculator were analyzed separately. The actual versus predicted lengths of stay were compared using Wilcoxon matched pairs analysis. For time to event analysis, a Kaplan Meyer analysis was performed using the logrank test. Results were considered significant if p < 0.05.

RESULTS

A total of 165 patients obtained from the prospectively maintained database of complications of patients undergoing PD for confirmed or suspected malignancy at Washington University School of Medicine (St. Louis, MO) over a two year period were included in the final analysis of this study. Both the standard and pylorus preserving Whipple procedure were included in the study and all underwent an end to side pancreaticojejunostomy. A total of 147 had a CPT code of 48150 (standard Whipple), and 18 had a CPT code of 48153 (pylorus-sparing Whipple).

The patient demographic and clinical preoperative information is presented in Table 1. Overall, the study group was 58% male (n=96) with a mean age of 63 years (median 64, range 30 to 90). The mean BMI (27.2) reflects an overall obese population. No patients had sepsis within 48 hours or ascites within 30 days prior to their procedure. A total of 62% (n=103) of patients had an American Society of Anesthesiologists (ASA) status of 3 (severe systemic disease) and the second-highest represented ASA status was 2 (n=55). A total of 146 patients had preoperative biopsies, 112 of which definitively confirmed malignancy, 5 were suspicious for malignancy, and 11 showed atypical cells. 15 were negative for malignancy. On final pathologic report 139 patients (84%) had malignancy confirmed on the pathology report.

Table 1.

Patient Demographics and Known Predictors of Morbidity/Mortality

Mean Age in years, mean +/− SD (range)	63.2 +/− 11.4 (30–90)
ASA classification, mean +/− SD	2.65 +/− 0.56
BMI (703*lbs/inches2), mean +/− SD (range)	27.20 +/− 5.29 (17–44)
Male Sex	58%
Hypertension	53%
Diabetes	30% (18% insulin dependent)
Coronary Artery Disease/Prior MI	15%
Current smoker (within 1 year)	36%
COPD	17%
Acute renal failure (pre-existing)	0%
Dyspnea with moderate exertion	9%
Dialysis	1%
Recent steroid use	8%
Functional status: independent	99%
Wound class: clean/contaminated	100%
Ventilator dependent	0
Disseminated cancer	0
Ascites	0
Sepsis	0
Malignant Disease	84%

The regression analysis to determine difference between actual incidence and estimated risk showed the actual incidence was significantly lower than the estimated risk of any complication (29.1% actual vs. 32.7% estimated, p=0.044), cardiac-related events (1.8% vs. 2.0 %, p=0.004), and surgical site infections (18.8% vs. 20.0%, p=0.015). For the other 5 of 8 complication types (serious complications, urinary tract infections, venous thromboembolism, death), there was no significant difference between observed and calculator predicted. Since there was zero actual incidence of renal failure of the 165 patients in the study, a statistical comparison by regression analysis of the actual incidence of zero to the mean estimated risk of 1.8% (with a standard deviation of 1.3%) could not be performed. Due to LOS being a continuous variable, a paired t-test was used and we found the NSQIP calculator predicted LOS to be significantly shorter than the actual duration of hospitalization following pancreatectomy (median 8 actual days vs. 8.5 predicted days, Actual-Estimate difference: p < 0.001) Table 2.

Table 2.

Comparison of Outcomes to Estimated Risk

Complication Type	Predicted % Risk (mean)	Actual % Incidence	P-Value*
Serious	22.1	25.5	0.311
Any	32.6	29.1	0.044
Pneumonia	4.8	5.5	0.111
CA/MI	2.0	1.8	0.004
SSI	20.0	18.8	0.015
UTI	4.5	4.8	0.333
VTE	2.5	0.6	0.379
Death	1.5	1.8	0.876
Length of Stay (Days)	8.5+	8+	<0.001**

^*Regression Analysis

^**t-test

⁺Median

The results of the calculator’s ability to predict individual outcomes in our test cohort, are shown in Table 3. For CA/MI, there was a significantly higher estimated risk in those who had those specific complications than those who did not. There was no significant difference in the mean estimate risk of serious complication, UTI, VTE and death.

Table 3.

Estimated Risk when separating Actual Complication Status into Yes/No

Variable	Actual		P-Value**
	Yes	No
Serious (n, mean est. ± SD)	42, 22.8 ± 4.5	123, 21.9 ± 5.0	0.134
Any (n, mean est. ± SD)	48, 34.5 ± 8.3	117, 31.9 ± 7.2	0.060
PNA (n, mean est. ± SD)	9, 6.5 ± 3.4	156, 4.7 ± 3.3	0.064
CA/MI (n, mean est. ± SD)	3, 4.8 ± 2.0	162, 2.0 ± 1.7	0.023
SSI (n, mean est. ± SD)	31, 21.9 ± 5.6	134, 19.6 ± 4.4	0.051
UTI (n, mean est. ± SD)	8, 5.2 ± 2.8	157, 4.5 ± 2.0	0.471
VTE (n, mean est. ± SD)	1, 3.3	164, 2.5 ± 0.9	0.261
Death	3, 1.4 ± 1.2	162, 1.6 ± 1.7	0.951

^**Kruskal-Wallis test

We sought to understand which factors were driving the observed increase in postoperative inpatient hospitalization. As pancreatectomy is a complex surgical intervention, we hypothesized that complications specific to this intervention may be responsible for this finding. In addition to the specific complications listed in the NSQIP ACS Risk calculator, there are procedure-specific complications that affect morbidity and mortality of the Whipple procedure. Analysis of the prospective pancreatectomy complications database at our institution showed that 38% of patients (62 of 165) in our study cohort experienced a pancreas-related complication defined as delayed gastric emptying, pancreatic fistula, ileus, or bleeding. There were 30 incidences of pancreatic fistula (18%), 22 of delayed gastric emptying (13%), 8 of ileus (5%), and 3 of postoperative, procedure-related bleeding (upper gastrointestinal and/or anastomotic bleed; 2%).

Next we explored if these pancreatectomy-specific complications were a driving influence behind the differences in predicted vs actual length of stay. Supporting our hypothesis, we found that patients having a pancreatectomy-related complication unaccounted for in the ACS-NSQIP Risk calculator had a significant increase in duration of hospitalization compared to that predicted by the NSQIP risk calculator (8.0 days predicted vs 12.2 days actual, p<0.0001, Figures 1,2), while patients having a complication included in the calculator model did not show this variation.

Figure 1.

Pancreatectomy specific* complications result in discrepancies between predicted & actual LOS following Whipple Procedure

DISCUSSION

The ACS-NSQIP calculator was designed to help patients and surgeons better understand and stratify operative risk and improve perioperative decision-making.⁽¹⁸⁾ Pancreatic adenocarcinoma is currently the 4^th leading cause of cancer related mortality⁽²³⁾ and PD offers the only potential chance for cure in this patient population. However, pancreatic resection remains a highly morbid operation, and in our continuous efforts to improve outcomes for patients with pancreatic cancer, we sought to assess the use of this calculator as an aide to informed consent in this patient population. Overall, our results are congruent with the estimates of general surgical complications following PD, which is not surprising, considering the ACS NSQIP calculator is well validated using a large, heterogeneous surgical patient population. Although some of our reported findings do show a statistically significant difference in the actual versus predicted complication rate, these findings are rather modest and unlikely to be clinically relevant.

Nonetheless, our results do show an intriguing finding when it comes to the length of stay following PD. Predicted LOS was significantly shorter than actual LOS (8.0 vs. 8.5 days) for our patient population. While a clinically insignificant difference of a median 0.5 days, we sought to explore the etiology behind this finding. Our work suggests that pancreatectomy-specific complications (pancreatic leaks, delayed gastric emptying, postoperative hemorrhage) are likely responsible for this discrepancy. Patients without a pancreatic-specific post-procedural complication had no difference in actual vs. predicted LOS. Conversely, patients with pancreatectomy-specific complications had both a statistically and clinically significant difference in LOS, with a 12.2 days actual vs. 8.0 days predicted LOS. This suggests that while the NSQIP calculator is well validated and excellent at offering relevant information regarding the likely length of stay in patients with the common complications seen in general surgery, procedural specific complications unique to PD may remain a significant source of variation in LOS from that predicted in the NSQIP risk calculator. This data signifies that the incorporation of such procedurally specific variables, such as those collected in the NSQIP Pancreatectomy Database, may present an optimal opportunity for creating better models for complex surgical interventions, such as the Whipple procedure.

The fact that 38% of patients in our cohort had a pancreatectomy specific complication highlights the requirement to discuss the likelihood of such events on an individual patient level. It is important to note that of these specific complications that were not accounted for by the calculator, pancreatic fistulas appear to be the most prevalent. Late deaths resulting from complications are well reported for this operation and are not currently captured in the 30 day NSQIP calculator predicted mortality rate.^{(24, 25)} While one might think that a pancreatic fistula is an organ-space infection (OSI), a recent study showed that the sensitivity and specificity of OSI in NSQIP are too low for OSI to be a sufficient surrogate for grade B and grade C pancreatic fistula. Procedure-specific variables are required for ACS NSQIP to improve outcomes.^{(21, 26)} Delayed gastric emptying, due to its frequency of occurrence and ability to affect immediate postoperative quality of life, cannot be ignored when discussing possible complications following PD.

In a patient population with such a poor nonsurgical prognosis as pancreatic cancer, where the necessity of the procedure is often a foregone conclusion, the utility of the ACS-NSQIP Surgical Risk Calculator as a consenting tool comes into question.⁽²⁷⁾ This poses unique challenges for the surgeon with regards to performing optimal informed consent as the risks of PD are often deemed acceptable due to the high mortality rate of pancreatic cancer. What does this risk calculator change in the context of cancer? It brings about a very unique form of informed consent. If the surgery presents the patient with the best chance of long-term survival, what good will advising the patient of a 5 or 10% chance of death versus a 2% chance provide? Or in the case of cardiac events, although significant, the difference is merely a half a percentage point. Lipkus et. al. demonstrated that even among a highly educated population, approximately 1/5^th of participants were unable to identify if 1%, 5%, or 10% represented the greatest risk.⁽²⁸⁾ For those with benign disease, 16% of the patients in our study, some data suggests that a PD should be avoided due to the complex nature of the procedure and the introduction of potential complications.⁽²⁴⁾ In the case of benign disease, a case-by-case assessment with the risk calculator to quantify whether the patient is above or below average risk for complications could effectively help decision-making prior to surgery.

An excellent benefit of the calculator is that it provides a simplified overview of a limited number of the common potential complications that may befall the patient postoperatively in a very easily understandable way. Paruch et. al. and Kinnier et. al. outline the importance of the NSQIP calculator as a tool in patient-centered care, particularly when consenting patients for elective oncologic procedures. When used as a consenting tool, it can help to make the patient aware of what symptoms may signify the early onset of a complication. The patients in our study left the hospital after an average of 9.6 days, leaving 20.4 more days under the “30 day postop” NSQIP complication umbrella for complications to occur. Therefore the discharge education may be tailored to a patient’s particular risks. As suggested by Paruch et. al., patients who know they have a higher risk of pneumonia may be more incentivized to actively use their spirometers and monitor their breathing.^{(29, 30)}

Possible sources of error in our study include the fact that we only include a total of 165 patients. This small sample size may mean that we did not accurately capture the true spectrum of complications at our center. Also, being at a tertiary referral center, our patient population may not accurately reflect that of many of the centers around the country performing PDs. We did not adjust the calculator for surgeon adjustment of risks, and doing so on a case-by-case basis may have further improved the accuracy of our risk calculations. We do acknowledge the existence of PD-specific morbidity and mortality calculators, such as that developed by Greenblatt et. al., which includes variables distinct from the ACS NSQIP risk calculator (e.g. weight loss, bleeding disorder, recent radiotherapy) in order to improve PD specific morbidity and mortality assessment.^(9,31) However, we did not examine the role of pancreatectomy specific risk calculators, as our focus was on assessing the utility of the ACS-NSQIP Surgical Risk Calculator.^(31–34)

Although we think that the addition of pancreas specific outcomes to the NSQIP risk calculator will increase its utility for pancreatic procedures, we do not think a risk calculator is the best way to discuss risk with a patient.^{(35, 36)} As mentioned earlier, it has been shown that up to 20% of a well-educated population is unable to identify if 1%, 5% or 10% is a greater risk.⁽²⁸⁾ Goldman’s index, developed in 1977, was one of the first tools to predict percentage chance of a postoperative complication (in this case cardiac risk).⁽³⁷⁾ Neuman et. al. discuss how statistical prediction tools, such as Goldman’s index, paint a picture that is extremely difficult for patients to reconcile. It was initially felt that giving a patient an exact percentage chance of risk for a given outcome was an improvement upon the classic classification of a patient as a “good” or “poor” surgical candidate.⁽³⁸⁾ Schwarze et. al. outlines an alternative method, whereby best and worst-case scenarios, along with the most likely scenario, are outlined for the patient.⁽³⁹⁾ This method provides an easier to grasp concept of what to expect post-surgery, and statistics can be used alongside this tool to further fill in knowledge gaps and to better paint an accurate picture. We feel that the utilization of more pancreas specific statistical outcomes, alongside a discussion based upon best and worst case scenarios, will provide a more understandable representation of how a patient will fare postoperatively after a Whipple procedure.

Figure 2.

Pancreatectomy specific* complications result in prolonged LOS & are not accounted for in NSQIP risk calculator