Abstract
Background
The morbidity of pancreatoduodenectomy remains high and the mortality may be significantly increased in high-risk patients. However, a method to predict post-operative adverse outcomes based on readily available clinical data has not been available. Therefore, the objective was to create a ‘Pancreatectomy Risk Calculator’ using the American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) database.
Methods
The 2005–2008 ACS-NSQIP data on 7571 patients undergoing proximal (n =4621), distal (n =2552) or total pancreatectomy (n =177) as well as enucleation (n =221) were analysed. Pre-operative variables (n =31) were assessed for prediction of post-operative mortality, serious morbidity and overall morbidity using a logistic regression model. Statistically significant variables were ranked and weighted to create a common set of predictors for risk models for all three outcomes.
Results
Twenty pre-operative variables were statistically significant predictors of post-operative mortality (2.5%), serious morbidity (21%) or overall morbidity (32%). Ten out of 20 significant pre-operative variables were employed to produce the three mortality and morbidity risk models. The risk factors included age, gender, obesity, sepsis, functional status, American Society of Anesthesiologists (ASA) class, coronary heart disease, dyspnoea, bleeding disorder and extent of surgery.
Conclusion
The ACS-NSQIP ‘Pancreatectomy Risk Calculator’ employs 10 easily assessable clinical parameters to assist patients and surgeons in making an informed decision regarding the risks and benefits of undergoing pancreatic resection. A risk calculator based on this prototype will become available in the future as on online ACS-NSQIP resource.
Keywords: ACS-NSQIP, pancreatectomy, risk calculator, pancreatic resections
Introduction
During the past decade, the number of pancreatic resections being performed at high volumes centres has progressively increased.1 This change is multifactorial and includes increased detection of malignant and premalignant pancreatic lesions, improved outcomes at regional referral institutions and surgeons' willingness to operate on older and higher-risk patients. The annual number of pancreatic resections has increased by 15% in the past 20 years, and resection for benign pancreatic disease has increased by 27%.2 However, the morbidity of pancreatic surgery remains high, and the mortality may be significantly increased in high-risk patients. With a narrow therapeutic margin, careful patient selection is imperative to minimize post-operative complications and operative mortality.
Single-institution studies have reported a low peri-operative mortality rate of 1–2% for these procedures, but these results are not always reproducible at other institutions.3–5 In contrast, population-based studies have reported a higher peri-operative mortality rate ranging from 4.6% to 7.8%.6,7 Morbidity after pancreatic resections still remains high with complication rates varying from 20% to 50%.8,9 Recently, The Nationwide Inpatient Sample (NIS) has been used to develop both a nonogram and a risk score to predict in-hospital mortality for cancer patients undergoing pancreatectomy.10,11 These NIS-based analyses give generalized estimates of national inpatient mortality rates for patients after pancreatic resections for cancer. However, they do not include estimates for other pancreatic diseases nor do they provide information about the risk for developing post-operative complications.
The American College of Surgeons-National Surgery Quality Improvement Program (ACS-NSQIP) currently collects data on pre-operative risk factors as well as post-operative morbidity and mortality to assess surgical quality at more than 200 hospitals. The NSQIP was first developed in Veterans Affairs hospitals in the 1990s12,13 and was then piloted in selected university medical centres in the early 2000s.14 As the ACS-NSQIP has evolved, the potential to provide robust outcomes on patients undergoing pancreatic surgery and to develop risk calculators has become a reality.15,16 Therefore, the objective of this analysis was to use the ACS-NSQIP database to develop a pancreatectomy risk calculator to predict post-operative adverse outcomes based on readily available clinical data.
Methods
ACS-NSQIP
The American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) is a prospective, multicentre clinical registry that was created to provide feedback on risk-adjusted outcomes to hospitals for quality-improvement purposes. The sampling strategy, data abstraction procedures, variables collected and structure have already been published.17–20 The programme collects detailed information on patient demographics, pre-operative risk factors and laboratory values, operative variables and post-operative events using standardized definitions.12 From the ACS NSQIP database for 1 January 2005 to 31 December 2008, patients >16 years of age who underwent a major pancreatic resection were identified using Current Procedural Terminology (CPT) codes. These data were used for the development of a pancreatectomy risk calculator.
Pre-operative variables
A set of potential predictive variables was constructed from ACS-NSQIP data fields. The patient demographic variables of age (<65, 65–74, 75–84 and >85 years) and gender, and the lifestyle factor of smoking status (within 1 year of operation) and alcohol status were considered. The pre-operative factors considered were American Society of Anesthesiologists (ASA) classification (I/II, normal healthy or mild systemic disease; III, severe systemic disease; IV/V, severe systemic disease that is a constant threat to life or moribund), pre-operative functional status (independent vs. partially or totally dependent), dyspnoea (none, moderate exertion, at rest) and body mass index (BMI) (normal, underweight, overweight, three levels of obesity as classified by the World Health Organization). Comorbidities considered were ventilator dependence, sepsis, a history of chronic obstructive pulmonary disease (COPD), hypertension requiring medication, current pneumonia, ascites, congestive heart failure (within 30 days prior to the procedure), coronary heart disease (includes angina, myocardial infarction within 30 days before the operation, percutaneous cardiac intervention, or coronary artery bypass surgery), peripheral vascular disease (includes revascularization for peripheral vascular disease, claudication, rest pain, amputation, or gangrene) and a neurological event or disease (includes stroke with or without residual deficit, transient ischaemic attack, haemiplegia, paraplaegia, quadriplaegia, or impaired sensation), diabetes (oral medication or insulin dependent), dialysis, acute renal failure, weight loss (>10% in past 6 months), bleeding disorders, current chemotherapy or recent radiotherapy, oesophageal varices, chronic steroid use and red blood cell transfusion prior to the procedure. All of these comorbidities are rigorously defined within the ACS-NSQIP. A total of 31 pre-operative variables were assessed for prediction of post-operative mortality, serious morbidity, or overall morbidity.
Outcomes
Outcomes were assessed at 30 days regardless of whether the patient was discharged, remained hospitalized or was admitted to a different institution. Outcomes included mortality (all-cause death within 30 days after the operation), serious morbidity and 30-day overall morbidity. Overall morbidity includes superficial surgical site infection (without pre-operative wound infection), deep incisional surgical site infection (without pre-operative wound infection), pneumonia (without pre-operative pneumonia), unplanned intubation (without pre-operative ventilator dependence), progressive renal insufficiency (without pre-operative renal failure or dialysis), urinary tract infection and deep venous thrombosis. Variables that were assigned to serious morbidity were organ space surgical site infection (without pre-operative wound infection), wound disruption, cerebrovascular accident or stroke, myocardial infarction, cardiac arrest requiring cardiopulmonary resuscitation, pulmonary embolism, ventilator dependence longer than 48 h (without pre-operative ventilator dependence), acute renal failure (without pre-operative renal failure or dialysis), bleeding complications defined by transfusions in excess of four units and sepsis or septic shock (if pre-operative sepsis exists, it must worsen post-operatively).
Risk calculator development
The risk calculator was developed after all pre-operative variables were made categorical and entered with CPT codes and ICD-9 (International Classification of Disease Codes, 9th edition) codes. CPT codes were used to categorize surgical procedure by extent or type (proximal pancreatectomy, distal pancreatectomy, total pancreatectomy or enucleation). The CPT codes used were 48140, 48145, 48146, 48150, 48152, 48153, 48154, 48155 and 48120. The diagnoses were categorized according to the indication using ICD-9 codes: acute pancreatitis, chronic pancreatitis, benign neoplasm, malignant neoplasm or other. Operations were also classified by emergent/non-emergent procedures and wound class (class 1/2: clean, clean/contaminated vs. class 3/4: contaminated or dirty/infected).
Pre-operative laboratory variables examined included haematocrit, white blood count, platelet count, sodium, blood urea nitrogen, creatinine, albumin, partial thromboplastin time and prothrombin. Values were categorized using ACS-NSQIP definitions of normal and abnormal, and missing data constituted a third categorical level, an indicator variable. Of note, all of the laboratory values were forced into the model and were found not to have a substantial impact on the overall determination of predictors for the risk calculator. Thus, they were not included in the final risk calculator model.
Variables that entered the model for mortality included: age group, systemic sepsis, functional health status, ASA classification, history of congestive heart failure, dyspnoea, previous or concurrent chemotherapy, oesophageal varices and type of surgery. The mortality model had four forced variables including gender, BMI classification, coronary heart disease and bleeding disorder. Variables that entered the predictive model for morbidity included: age group, gender, BMI classification, systemic sepsis, functional status, ASA classification, surgical extent, coronary heart disease, history of severe COPD, smoking status, dyspnoea, bleeding disorders and weight loss greater than 10%.
Statistical analysis
All variables were converted to a categorical format. The statistical model was constructed in two stages. First, forward stepwise logistic regression models for mortality, serious morbidity and overall morbidity were run which included all the independent variables mentioned above.21 This modelling method adds variables to the model when they provide significant, independent contributions. A second step, the Firth penalized likelihood approach, was needed in order to account for some predictors that have empty cells as a result of a low occurrence rate or small sample sizes.22 For example, mortality is an infrequent outcome and, when crossed with certain predictors, may result in a cell that has no mortality outcome under at least one level of the predictor. The presence of these empty cells can compromise the validity of the ordinary logistic model fitting algorithm. Firth's penalized likelihood approach was therefore used to achieve model consequence under these empty cell conditions.22 However, as Firth's method precludes step-wise methods, a two-step process was used where step-wise selected variables were forced into Firth-adjusted models for final parameter specifications.
Model quality was evaluated using Hosmer–Lemeshow goodness-of-fit tests for calibration (correspondence in predictions and observations across the range of predictions) and c-statistics for discrimination.23,24 The c-statistic was considered the most relevant measure of model success and refers to the ability of the risk estimate to discriminate cases from non-cases. If discrimination is no better than chance, the c-statistic will equal 0.50. All analysis and data manipulation were done using SAS 9.2 (SAS Institute, Inc., Cary, NC, USA).
Patient examples
To test the model and determine if it gave results similar to nationally published data, we developed three example patients who fell into low-, intermediate- and high-risk areas. The low-risk patient was a 45-year-old female with a mucinous cyst neoplasm (MCN) in the neck of the pancreas. This theoretical patient had an ASA class of II, a BMI of 25 and no pre-operative sepsis. She is now fully functional with no coronary heart disease, no dyspnoea or bleeding disorders. The pre-operative plan is enucleation of the MCN. The intermediate risk patient was a 75-year-old male with an intraductal papillary mucinous neoplasm (IPMN) in the tail of the pancreas. He has a BMI of 35, an ASA class of III, no pre-operative sepsis and is fully functional. He had a prior coronary artery bypass graft but did not have dyspnea or bleeding disorders. He is scheduled for a laparoscopic distal pancreatectomy. The high-risk patient was a 55-year-old male with chronic pancreatitis as a result of excess alcohol intake. He has chronic pain requiring narcotics and a distal bile duct stricture which was stented endoscopically. This patient has a BMI of 30, sepsis because of cholangitis, is dependent in an intensive care unit, has an ASA class of IV, had a prior coronary angioplasty, dyspnea with moderate exertion and has a bleeding disorder. His surgeon is contemplating a pancreatoduodenectomy for relief of pain and biliary obstruction.
Results
Patient characteristics
The 2005 to 2008 ACS-NSQIP dataset yielded 7571 pancreatic procedures at 193 hospitals (Table 1). The average patient age was 61.9 years with 47.7% being male. ASA Class III was most common (60.8%). The most frequent procedure was a proximal pancreatectomy at 61.0%, and the most common indication was neoplasm at 66.5%. Mortality was 2.5%, serious morbidity was 21.2% and overall morbidity was 31.8%.
Table 1.
Patient demographics in ACS-NSQIP used for pancreatectomy risk calculator
| Variable | 2005–2008 |
|---|---|
| n | 7571 |
| Hospitals, n | 193 |
| Cases/hospital, n (range) | 39.2 (1–519) |
| Age, years,a mean ± SD | 61.9 ± 13.8 |
| Gender, % male | 47.7 |
| ASA, % | |
| I/II. Normal healthy/Mild systemic disease | 34.7 |
| III. Severe systemic disease | 60.8 |
| IV/V. Severe systemic disease/Moribund | 4.5 |
| Surgical extent, % | |
| Proximal pancreatectomy | 61.0 |
| Distal pancreatectomy | 33.7 |
| Enucleation | 2.9 |
| Total pancreatectomy | 2.3 |
| Indication for surgery, %b | |
| Malignant neoplasm | 66.5 |
| Benign neoplasm | 18.5 |
| Chronic pancreatitis | 5.0 |
| Acute pancreatitis | 1.3 |
| Other | 8.7 |
| Outcomes | |
| Mortality | 2.5 |
| Serious morbidity | 21.2 |
| Overall morbidity | 31.8 |
Age recorded as 90+ had been recorded to 90.
Taken from the reported ICD-9 code for post-operative diagnosis in ACS NSQIP.
ACS NSQIP, American College of Surgeons National Quality Improvement Program; ASA, American Society of Anesthesiologists Physical Status Classification.
The individual mortality rates for the procedures were 2.9% for proximal pancreatectomy, 1.7% for distal pancreatectomy, 0.4% for pancreatic enucleation and 4.8% for total pancreatectomy.
Risk calculator
Application of the variable selection process for the dataset yielded six variables that appeared in models for all three outcomes and five variables that appeared in two outcomes (Table 2). One variable, coronary heart disease, was only involved in one model but had significant impact and early selection and therefore was considered to be important enough to be entered into the dataset. Based on entry into individual models, the 10 variables which were most highly weighted and found to have the highest rank in all three models, were chosen as the universal data set to develop the pancreatic risk calculator. Odds ratios for the variables selected for the universal model showed findings consistent with clinical expectations (Table 3). The relative odds ratios for age and surgical extent are illustrated in Fig. 1a,b, respectively. Models using the 10-variable universal dataset had acceptable discrimination and calibration for each outcome (Table 4). The c-statistics for mortality, serious morbidity and overall morbidity were 0.74, 0.61 and 0.61, respectively, and the Hosmer–Lemoshow fit statistics were not significant, indicating that the model had adequate fit for the variables entered.
Table 2.
Variables selected in construction of the pancreatectomy risk calculator
| ACS NSQIP variablesa | Mortalityb | Serious morbidityb | Overall morbidityb | Models, n | Included in universal model |
|---|---|---|---|---|---|
| ASA classification | 2 | 2 | 1 | 3 | Yes |
| Functional health status | 1 | 3 | 4 | 3 | Yes |
| Sepsis | 6 | 1 | 2 | 3 | Yes |
| Surgical extent | 5 | 6 | 3 | 3 | Yes |
| Age group | 3 | 8 | 8 | 3 | Yes |
| Dyspnoea | 4 | 5 | 11 | 3 | Yes |
| Body mass index | 7 | 7 | 2 | Yes | |
| Coronary heart disease | 5 | 1 | Yes | ||
| Gender | 4 | 12 | 2 | Yes | |
| Bleeding disorder | 11 | 6 | 2 | Yes | |
| Oesophageal varices | 8 | 13 | 2 | No | |
| COPD | 12 | 9 | 2 | No | |
| Congestive heart failure | 7 | 1 | No | ||
| Chemotherapy | 9 | 1 | No | ||
| Wound class | 9 | 1 | No | ||
| Peripheral vascular disease | 10 | 1 | No | ||
| Smoking status | 10 | 1 | No | ||
| Weight loss | 13 | 1 | No | ||
| Ascites | 14 | 1 | No | ||
| Neurological disease | 14 | 1 | No | ||
Some variables have been restructured.
Selection order.
ACS NSQIP, American College of Surgeons National Quality Improvement Program; ASA, American Society of Anesthesiologists Physical Status Classification; COPD, chronic obstructive pulmonary disease.
Table 3.
Percentages, odds ratios and confidence intervals for variables in pancreatectomy risk calculator
| ACS NSQIP variablesa | % of patients | Mortality | Serious morbidity | Overall morbidity |
|---|---|---|---|---|
| Age (<65 years) | 53.3 | |||
| 65–74 years | 27.1 | 1.70 (1.17–2.47)b | 1.10 (0.96–1.26)b | 1.13 (1.00–1.27)b |
| 75–84 years | 17.6 | 2.28 (1.54–3.38) | 1.30 (1.11–1.51) | 1.26 (1.10–1.45) |
| ≥85 years | 2.0 | 3.54 (1.77–7.05) | 1.57 (1.08–2.27) | 1.78 (1.27–2.49) |
| Gender Male | 47.7 | 1.16 (0.86–1.57) | 1.22 (1.09–1.37)b | 1.13 (1.02–1.25)c |
| Body mass index (normal) | 36.4 | |||
| Underweight | 3.4 | 0.96 (0.40–2.33) | 0.94 (0.67–1.31)b | 0.95 (0.71–1.26)b |
| Overweight | 34.8 | 1.29 (0.91–1.83) | 1.15 (1.00–1.31) | 1.16 (1.02–1.30) |
| Class 1 obesity | 15.7 | 1.28 (0.82–2.02) | 1.34 (1.13–1.58) | 1.34 (1.16–1.56) |
| Class 2 obesity | 6.1 | 0.91 (0.44–1.91) | 1.22 (0.96–1.56) | 1.24 (1.00–1.53) |
| Class 3 obesity | 3.7 | 2.32 (1.17–4.63) | 1.83 (1.38–2.44) | 1.60 (1.23–2.09) |
| Sepsis | 3.0 | 2.62 (1.30–3.95)b | 2.26 (1.70–3.01)b | 2.10 (1.60–2.79)b |
| Functional health status (dependent) | 3.3 | 3.27 (2.05–5.21)b | 1.73 (1.30–2.29)b | 1.75 (1.33–2.30)b |
| ASA classification I/II (no/mild disturbance) | 34.7 | |||
| Class III (severe disturbance) | 60.8 | 2.33 (1.49–3.64)b | 1.18 (1.03–1.34)c | 1.20 (1.06–1.33)b |
| Life-threatening/moribund | 4.5 | 3.20 (1.70–6.03) | 1.33 (1.01–1.75) | 1.40 (1.09–1.80) |
| Coronary heart disease | 10.8 | 1.18 (0.80–1.73) | 1.20 (1.01–1.43)c | 1.26 (1.08–1.48)b |
| Dyspnoea | 9.2 | |||
| Moderate exertion | 8.4 | 1.72 (1.15–2.58)b | 1.38 (1.14–1.66)b | 1.36 (1.14–1.61)b |
| At rest | 0.8 | 4.70 (2.15–2.58) | 1.44 (0.82–2.53) | 1.08 (0.63–1.86) |
| Bleeding disorder | 2.9 | 1.15 (0.58–2.29) | 1.43 (1.06–1.94)c | 1.68 (1.27–2.23)b |
| Surgical extent (proximal) | 61.0 | |||
| Distal pancreatectomy | 33.7 | 0.62 (0.43–0.88)b | 0.77 (0.68–0.88)b | 0.73 (0.66–0.82)b |
| Enucleation | 2.9 | 0.10 (0.01–1.52) | 0.74 (0.52–1.06) | 0.63 (0.46–0.87) |
| Total pancreatectomy | 2.3 | 1.86 (0.91–3.79) | 1.07 (0.74–1.53) | 0.91 (0.66–1.27) |
Some variables have been restructured.
P-values < 0.01.
P-values < 0.001 for the variable.
ACS NSQIP, American College of Surgeons National Quality Improvement Program; ASA, American Society of Anesthesiologists Physical Status Classification.
Figure 1.
(a) Odds ratios for increasing patient age in predicting mortality. (b) Odds ratios for the four pancreatectomy procedures
Table 4.
Model performance for pancreatectomy risk calculator
| Model performance | Mortality | Serious morbidity | Overall morbidity |
|---|---|---|---|
| Rate (n, %) | 186 (2.5) | 1605 (21.2) | 2411 (31.8) |
| C-statistic | 0.74 | 0.61 | 0.61 |
| Hosmer–Lemeshow | 0.28 | 0.61 | 0.79 |
Patient examples
The patient examples that were entered into the model showed levels of mortality, serious morbidity and overall morbidity comparable to most literature on this topic (Table 5, Fig. 2). The low-risk patient who had no abnormal variables and received a pancreatic enucleation for a benign neoplasm had a risk of mortality, serious morbidity and overall morbidity of 0.08%, 12.6% and 18.6%, respectively. The intermediate-risk patient who had four abnormal variables and received a distal pancreatectomy for an IPMN had a 2.1% risk of mortality. The risk for serious morbidity and overall morbidity were higher at 23% and 32.6%. The high-risk patient who had eight abnormal variables and was being considered for a proximal pancreatectomy had a mortality risk of 33.6%, with a risk of serious morbidity at 77.2% and overall morbidity at 87.6%.
Table 5.
Patients tested with pancreatectomy risk calculator
| Patient performance | Mortality | Serious morbidity | Overall morbidity |
|---|---|---|---|
| Low risk | 0.08% | 12.6% | 18.6% |
| Intermediate risk | 2.1% | 23.0% | 32.6% |
| High risk | 33.6% | 77.2% | 87.6% |
Figure 2.
Outcomes in low-, intermediate- and high-risk patients
Discussion
The ACS-NSQIP dataset from 2005–08 had 7571 patients who had undergone a pancreatic resection. Thirty-one preoperative variables were analysed for prediction of post-operative mortality, serious morbidity and overall morbidity using logistic regression models. Twenty pre-operative variables were found to be statistically significant. Ten out of the 20 risk factors were employed to produce mortality and morbidity risk models. The risk factors included age >74 years, male gender, BMI higher than 40, pre-operative sepsis, dependent functional status, ASA class more than II, history of coronary heart disease, dyspnoea on moderate exertion, a bleeding disorder and the contemplated procedure (Table 6). All of these variables can be easily assessed at the time of initial presentation and entered into the model so that a surgeon can provide an accurate assessment of operative risk, and a patient can receive individualized estimates of the risk of mortality, serious morbidity and overall morbidity.
Table 6.
Risk factors for increased mortality, serious morbidity and overall morbidity
| Age > 74 |
| Gender Male |
| BMI > 40 |
| Preoperative sepsis |
| Dependent functional health status |
| ASA classification > II |
| Coronary heart disease |
| Dyspnoea on moderate exertion |
| Bleeding disorder |
| Proximal or total pancreatectomy |
BMI, body mass index.
Patients who present with pancreatic pathology are often elderly and have multiple medical comorbidities. During initial evaluation, while performing a patient's history and physical, the comorbidities in the Pancreatectomy Risk Calculator can be identified and can be used to determine if the patient is an appropriate candidate for surgery. During this pre-operative counselling, the risk and benefits of the procedure are explained, and consent is obtained. Currently, the pre-operative counselling needed to obtain informed consent includes knowledge of published peri-operative morbidity and mortality rates. However, the majority of this information comes from referral centres and is not individualized for a particular patient. The Pancreatectomy Risk Calculator that was developed from the ACS-NSQIP database uses variables that can easily be discovered by a careful history and physical examination and will be available online as an ACS-NSQIP resource. This information also may be employed to plan resources for the patient and might be used to encourage risk stratification.
Predictive models that calculate the risk of post-operative mortality after pancreatectomy for cancer have been developed using the Nationwide Inpatient Sample.10,11 The NIS is managed by the Healthcare Cost and Utilization Project and is the largest all-payer database of hospital discharges, providing a 20% stratified sample of all non-federal hospitals in a given year. The data for the hospital and patients are entered retrospectively after discharge of the patient using ICD-9 codes. The integer-based risk score that was developed from the NIS uses age, gender, Charlson comorbidity score, pancreatectomy type and hospital volume as its predictors for in-patient mortality for patients with pancreatic adenocarcinoma.10 This model may be able to predict in-patient mortality; however, it does not have the ability to predict serious morbidity and overall morbidity. With published reports showing that morbidity for pancreatic resections varies from 20% to 50%, the chance of developing a post-operative complication is important for patients to appreciate prior to pancreatic resection.25
Nonograms are graphical devices or models that use algorithms or mathematical formulae to estimate the probability of an outcome for each individualized patient. The benefit of post-operative nonograms in predicting long-term survival has been proven in patients with cancers of various organ systems. Recently, a nonogram has been developed to preoperatively predict in-patient mortality for patients after pancreatic resection using the Nationwide Inpatient Sample (NIS).10 This nonogram has similar limitations as the integer-based risk score. Important factors that might contribute to peri-operative risk such as ASA class, functional status, weight loss, coronary heart disease and serum albumin are not available in the NIS. Furthermore, the nonogram, while web accessible, has numerous questions that need to be answered, making it lengthy and cumbersome for use. In comparing the ACS-NSQIP Pancreatectomy Risk Calculator to the NIS Risk Score and Nonogram, the Risk Calculator employed more patients (7571 vs. 5481 or 5715), over a shorter time (4 vs. 6 or 9 years), with more diagnoses (all vs. cancer) and procedures (all vs. major).
In recent years, considerable attention has been given to using computer-based methods to classify medical data to help predict outcomes. The general approach has been to develop computer algorithms that learn decision characteristics for data classification and then use them to classify future patients with unknown disease states or therapy outcomes. Several quantitative models ranging from simple linear analysis to more complex logistic regression and artificial neural networks (ANN) have been proposed. ANN is based on finding an optimal path from the sample space to the decision space. This process involves feeding unique input samples (features) and the matching responses (outcomes) to let the network learn from the examples and compose a map that inter-relates inputs to outputs through a complex set of interconnecting pathways or operations.26 Unlike logistic regression, which fits the data to a descriptive function, ANN transforms the data on each layer, changing its dimensional space to define the rule to get to the decision region. Thus, the two approaches are inherently different, raising the question if one approach has better diagnostic performance than the other. A meta-analysis comparing ANN with regression models in 28 studies found that both modes have similar performance.27 In another study, Dreiseitl surveyed 72 articles comparing ANN with logistic regression and found there was no difference in models for predicting outcomes.28 Currently, only one paper uses ANN for predicting clinical outcomes in patients with acute biliary pancreatitis and none for pancreatic surgery.29
A recent analysis of complication rates after pancreatectomy again employing the Nationwide Inpatient Sample also has been published.30 This analysis reported a 22.7% complication rate after pancreatectomy with no change over the 9 years of the analysis (1998–2006). Independent predictors of complications include age >74 years, total pancreatectomy, and low hospital resection volume. In comparison, the ACS-NSQIP Pancreatectomy risk Calculator provides robust information for prediction of both serious and overall morbidity with more recent clinical rather than administrative data.
The potential benefit of undergoing resection at high-volumes centres has led to regionalization of care for patients with pancreatic malignancies.1,31–33 However, contradictory data exist as to what defines high volume and whether volume should be defined by surgeon volume or hospital volume.34,35 A study using the Nationwide Inpatient Sample showed that volume alone accounted for less than 2% of the variance in peri-operative mortality after pancreatic resection.35 Furthermore, the Nationwide Inpatient Sample only provides information on hospital volume but is not able to provide surgeon specific data. The ACS-NSQIP has the potential to adjust outcomes by hospital volume and individual surgeon. Analyses of these options are underway but are not represented in the present. In this study of the 193 hospitals, 60% were academic/teaching institutions which could be described as high-volume hospitals for pancreatic resections. Furthermore, the average number of cases per hospital was 39 with the range from 1 case to 519 cases over that time period.
This study has several limitations. The ACS-NSQIP database only includes information from 2005–08 and from fewer than 200 hospitals nationwide. However, these institutions perform the majority of pancreatic surgery in the United States. In the 4-year period, 7571 pancreatic resections were performed and included in the development of the pancreatic risk calculator. Because of the limited number of pancreatic resections, the pancreatic risk model has not been validated with any subsequent data. The present plan is to carry out analyses of 2009 and future data for validation of this model. Being able to further evaluate both hospital- and surgeon-specific data would help to further determine patient outcome. Another limitation of the present study is the inability to define the frequency and morbidity related to operation-specific complications. For example, a major contributor to surgical site infection after pancreatectomy is pancreatic leak. A pancreatic fistula occurs in approximately 10% to 15% of pancreatoduodenectomy36 and 30% of distal pancreatectomy patients.37,38 In this current dataset, pancreatic fistula is grouped with other organ space infections. However, in an upcoming update to the ACS-NSQIP data structure pancreatectomy-specific pre-operative risk factors and post-operative outcomes will be collected. Having data on pancreatectomy-specific outcomes such as pancreatic fistula will be very helpful in determining the cause and improving outcomes.
Despite these limitations, the purpose of the present study was to develop a Pancreatectomy Risk Calculator using variables that are easily obtainable by history and physical examination. The Pancreatectomy Risk Calculator is not intended to substitute for surgeon judgment or experience but should be used as an additional resource in counselling patients who are being considered for high-risk pancreatic surgery. In the future ACS-NSQIP data will also provide increasingly specific data at the hospital-, surgeon- and procedure-specific levels. In summary, the ACS-NSQIP Pancreatectomy Risk Calculator employs 10 easily accessible clinical parameters to assist patients and surgeons in making an informed decision regarding risks and benefits of pancreatic resection. This system also may be helpful in resource planning and risk modification. A risk calculator based on this prototype will soon become available as an online ACS-NSQIP resource.
Disclosure of interest
None declared.
References
- 1.Zeigler KM, Nakeeb A, Pitt HA, Schmidt CM, Bishop SN, Moreno J, et al. Pancreatic surgery: evolution of a high volume center. Surgery. 2010 doi: 10.1016/j.surg.2010.07.029. in press. [DOI] [PubMed] [Google Scholar]
- 2.Teh SH, Diggs BS, Deveney CW, Sheppard BC. Patient and hospital characteristics on the variance of perioperative outcomes for pancreatic resection in the United States. Arch Surg. 2009;144:713–717. doi: 10.1001/archsurg.2009.67. [DOI] [PubMed] [Google Scholar]
- 3.Winter JM, Cameron JL, Campbell KA, Arnold MA, Chang DC, Coleman J, et al. 1423 pancreaticoduodenectomies for pancreatic cancer: a single institution experience. J Gastrointestinal Surg. 2006;10:1199–1210. doi: 10.1016/j.gassur.2006.08.018. [DOI] [PubMed] [Google Scholar]
- 4.Cameron JL, Riall TS, Coleman J, Belcher KA. One thousand consecutive pancreaticoduodenectomies. Ann Surg. 2006;244:10–15. doi: 10.1097/01.sla.0000217673.04165.ea. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Vin Y, Sima CS, Getrajdman GI, Brown KT, Covey A, Brennan MF, et al. Management and outcomes of postpancreatectomy fistula, leak and abscess: results of 908 patients resected at a single institution between 2000 and 2005. J Am Coll Surg. 2008;207:490–498. doi: 10.1016/j.jamcollsurg.2008.05.003. [DOI] [PubMed] [Google Scholar]
- 6.McPhee JT, Hill JS, Whalen GF, Zayaruzny M, Litwin DE, Sullivan ME, et al. Perioperative mortality for pancreatectomy: a national perspective. Ann Surg. 2007;246:246–253. doi: 10.1097/01.sla.0000259993.17350.3a. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ghaferi AA, Birkmeyer JD, Dimick JB. Variation in hospital mortality associated with inpatient surgery. New Engl JMed. 2009;361:1368–1375. doi: 10.1056/NEJMsa0903048. [DOI] [PubMed] [Google Scholar]
- 8.Yeo CJ, Cameron JL, Sohn TA, Lillemoe KD, Pitt HA, Talamini MA, et al. Six hundred fifty consecutive pancreaticoduodenectomies in the 1990s: pathology, complications, and outcomes. Ann Surg. 1997;226:248–257. doi: 10.1097/00000658-199709000-00004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Buchler MW, Friess H, Muller MW, Wheatley AM, Beger HU. Randomized trial of duodenum-preserving pancreatic head resection versus pylorus-preserving Whipple in chronic pancreatitis. Am J Surg. 1995;169:65–70. doi: 10.1016/s0002-9610(99)80111-1. [DOI] [PubMed] [Google Scholar]
- 10.Are C, Afuh C, Ravipati L, Sasson A, Ullrich F, Smith L. Preoperative nomogram to predict risk of perioperative mortality following pancreatic resections for malignancy. J Gastrointest Surg. 2009;13:2152–2162. doi: 10.1007/s11605-009-1051-z. [DOI] [PubMed] [Google Scholar]
- 11.Hill JS, Zhou Z, Simons JP, et al. A simple risk score to predict in-hospital mortality after pancreatic resection for cancer. Ann Surg Onc. 2010;17:1802–1807. doi: 10.1245/s10434-010-0947-x. [DOI] [PubMed] [Google Scholar]
- 12.Khuri SF, Henderson WG, Daley J, Jonasson O, Jones RS, Campbell DA, Jr, et al. Successful implementation of the Department of Veterans Affairs National Surgical Quality Improvement Program in the private sector: the Patient Safety in Surgery Study. Ann Surg. 2008;248:329–336. doi: 10.1097/SLA.0b013e3181823485. [DOI] [PubMed] [Google Scholar]
- 13.Khuri SF, Daley J, Henderson W, Hur K, Demakis J, Aust JB, et al. The Department of Veterans Affairs' NSQIP: the first national, validated, outcome based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg. 1998;228:491–507. doi: 10.1097/00000658-199810000-00006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Glasgow RE, Jackson HH, Neumayer L, Schifftner TL, Khuri SF, Henderson WG, et al. Pancreatic resection in Veterans Affairs and selected university medical centers: results of the patient safety in surgery study. J Am Coll Surg. 2007;204:1252–1260. doi: 10.1016/j.jamcollsurg.2007.03.015. [DOI] [PubMed] [Google Scholar]
- 15.Pitt HA, Kilbane M, Strasberg SM, Pawlik TM, Dixon E, Zyromski NJ, et al. ACS-NSQIP has the potential to create an HPB-NSQIP option. HPB (Oxford) 2009;11:405–413. doi: 10.1111/j.1477-2574.2009.00074.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Cohen ME, Bilimora KY, Ko CY, Hall BL. Development of an American College of Surgeons National Surgery Quality Improvement Program: morbidity and mortality risk calculator for colorectal surgery. J Am Coll Surg. 2009;208:1009–1016. doi: 10.1016/j.jamcollsurg.2009.01.043. [DOI] [PubMed] [Google Scholar]
- 17.ACS-NSQIP participant use file user's guide. https://acsnsqip.org/puf/docs/ACS_NSQIP_Participant_User_Data_File_User_Guide.pdf [accessed on 12 April 2010]
- 18.ACS-NSQIP program specifics: surgical case inclusion/exclusion overview. http://acsnsqip.org/main/program_case_inclusion_exclusion.asp [accessed on 12 April 2010]
- 19.ACS-NSQIP program specifics: surgical clinical nurse reviewer training. http://acsnsqip.org/main/program_nurse_training.asp [accessed on 12 April 2010]
- 20.ACS-NSQIP program specfics: ACS NSQIP data; participant use data file. http://acsnsqip.org/puf/PufRequestHomepage.aspx [accessed on 12 April 2010]
- 21.Anderson RP, Jin R, Grunkemeier GL. Understanding logistic regression analysis in clinical reports: an introduction. Ann Thorac Surg. 2003;75:753–757. doi: 10.1016/s0003-4975(02)04683-0. [DOI] [PubMed] [Google Scholar]
- 22. Firth penalized likelihood approach described at http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/statug_logistic_sect026.htm#statug.logistic.logisticdfirth [accessed on 12 April 2010]
- 23.Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: the Hosmer-Lemeshow test revisited. Crit Care Med. 2007;35:2052–2056. doi: 10.1097/01.CCM.0000275267.64078.B0. [DOI] [PubMed] [Google Scholar]
- 24.Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
- 25.Simons JP, Shah SA, Ng SC, Whalen GF, Tseng JF. National complication rates after pancreatectomy: beyond mere mortality. J Gastrointest Surg. 2009;13:1798–1805. doi: 10.1007/s11605-009-0936-1. [DOI] [PubMed] [Google Scholar]
- 26.Song JH, Venkatesh SS, Conant EA, Arger PH, Sehgal CM. Comparative analysis of logistic regression and artificial neural network for computer-aided diagnosis of breast masses. Acad Radiol. 2005;12:487–495. doi: 10.1016/j.acra.2004.12.016. [DOI] [PubMed] [Google Scholar]
- 27.Sargent DJ. Comparison of artificial neural networks with other statistical approaches: results from medical data sets. Cancer. 2001;91:1636–1642. doi: 10.1002/1097-0142(20010415)91:8+<1636::aid-cncr1176>3.0.co;2-d. [DOI] [PubMed] [Google Scholar]
- 28.Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform. 2002;35:352–359. doi: 10.1016/s1532-0464(03)00034-0. [DOI] [PubMed] [Google Scholar]
- 29.Yoldas O, Koc M, Karakose N, Kilic M, Tez M. Prediction of clinical outcomes using artificial neural networks for patients with acute biliary pancreatitis. Pancreas. 2008;36:90–92. doi: 10.1097/MPA.0b013e31812e964b. [DOI] [PubMed] [Google Scholar]
- 30.Lieberman MD, Kilburn H, Lindsey M, Brennan MF. Relation of perioperative deaths to hospital volume among patients undergoing pancreatic resection for malignancy. Ann Surg. 1995;222:638–645. doi: 10.1097/00000658-199511000-00006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Fong Y, Gonen M, Rubin D, Radzyner M, Brennan MF. Long term survival is superior after resection for cancers in high-volume centers. Ann Surg. 2005;242:540–544. doi: 10.1097/01.sla.0000184190.20289.4b. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Birkmeyer JD, Siewers AE, Finlayson EV, Stukel TA, Lucas FL, Batista I, et al. Hospital volume and surgical mortality in the United States. N Engl J Med. 2002;346:1128–1137. doi: 10.1056/NEJMsa012337. [DOI] [PubMed] [Google Scholar]
- 33.Riall TS, Nealon WH, Goodwin JS, Townsend CM, Jr, Freeman JL. Outcomes following pancreatic resection: variability among high-volume providers. Surgery. 2008;144:133–140. doi: 10.1016/j.surg.2008.03.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Nathan H, Cameron JL, Choti MA, Schulick RD, Pawlik TM. The volume-outcomes effect in hepato-pancreato-biliary surgery: hospital versus surgeon contributions and specificity of the relationship. J Am Coll Surg. 2009;208:528–538. doi: 10.1016/j.jamcollsurg.2009.01.007. [DOI] [PubMed] [Google Scholar]
- 35.Meguid RA, Ahuja N, Chang DC. What constitutes a ‘high volume’ hospital for pancreatic resection? J Am Coll Surg. 2008;206:622–631. doi: 10.1016/j.jamcollsurg.2007.11.011. [DOI] [PubMed] [Google Scholar]
- 36.Schmidt CM, Powell ES, Yiannoutsos CT, Howard TJ, Wiebke EA, Wiesenauer CA, et al. Pancreaticoduodenectomy: a 20-year experience in 516 patients. Arch Surg. 2004;139:718–725. doi: 10.1001/archsurg.139.7.718. discussion 725–727. [DOI] [PubMed] [Google Scholar]
- 37.Rodríguez JR, Germes SS, Pandharipande PV, Gazelle GS, Thayer SP, Warshaw AL, et al. Implications and cost of pancreatic leak following distal pancreatic resection. Arch Surg. 2006;141:361–365. doi: 10.1001/archsurg.141.4.361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Fahy BN, Frey CF, Ho HS, Beckett L, Bold RJ. Morbidity, mortality, and technical factors of distal pancreatectomy. Am J Surg. 2002;183:237–241. doi: 10.1016/s0002-9610(02)00790-0. [DOI] [PubMed] [Google Scholar]