Nomogram to Predict Postoperative Readmission in Patients Who Undergo General Surgery

Abstract

Importance

CMS has implemented penalties for hospitals with above average readmission rates under the hospital readmission reduction program. These changes will likely be extended to affect post-operative readmissions in the future.

Objective

The aim of this study was to identify variables which place patients at risk for readmission, develop a predictive nomogram, and validate this nomogram.

Design

A predictive nomogram was developed using the linear predictor method. This nomogram was validated prospectively in general surgery patients. Area under the curve and positive and negative predictive values were calculated.

Setting

This study was performed at a single academic institution using the ACS NSQIP database paired with institutional billing data.

Participants

Patients who underwent non-emergent, inpatient, general surgery procedures were included. The nomogram was developed in 2,799 patients and prospectively validated in 255 patients.

Main Outcome Measure

The primary outcome of interest was readmission within 30 days of discharge following an index hospitalization for a surgical procedure.

Results

Bleeding disorder, long operative time, in hospital complications, dependent functional status, and need for higher level of care at discharge independently predicted readmission. The nomogram accurately predicted readmission (c statistic = 0.756) in a prospective evaluation. The negative predictive value was 97.9% in the prospective validation, while the positive predictive value was 11.1%.

Conclusions

Development of an online calculator utilizing this predictive model will allow us to identify patients at high risk for readmission at the time of discharge. Patients with increased risk may benefit from more intensive post-operative follow up in the outpatient setting.

Introduction

Preventing hospital readmissions has become a national priority given the prevalence of readmissions and new legislation penalizing hospitals with high risk adjusted rates of readmission. Post-operative readmissions are common and have been found to range from 4–25% in general surgery patients.(1–9) A review of the medical and surgical literature found that preventable readmissions account for 9–50% of all readmissions.(8, 10) Although the use of hospital readmission rates as a measure of quality remains controversial,(9–11) the Center of Medicare and Medicaid Services (CMS) has tied hospital reimbursement with readmissions. As of 2010, CMS has reduced reimbursement to hospitals with higher than expected readmission rates.(12–14) While these changes do not currently affect surgical patients, it is only a matter of time before post-operative readmission face the same reimbursement penalties. As a result, identifying patients at high risk for readmission and implementing quality improvement projects aimed at decreasing readmissions has become a significant priority.

While algorithms for calculating readmission risk are common in the medical literature,(15) the surgical literature has focused more on general scoring systems for postoperative morbidity and mortality. The Physiologic and Operative Severity Score for the enumeration of Mortality and Morbidity (POSSUM) scoring system has been found to predict risk of morbidity and mortality in a variety of surgical patients.(16–19) unfortunately, the POSSUM scoring system cannot be used to evaluate hospital readmission separate from other morbidity. In vascular, thoracic and general surgery patients, a model using LOS and ASA class was found to be predictive of readmission within 30 days of surgery.(20) However, the authors did not evaluate general surgery patients independently.

In this study, we sought to develop and validate a nomogram predictive of readmission within 30 days from hospital discharge. Our study aims were: 1. To identify risk factors for post-operative readmission in general surgery patients, 2. Create a predictive nomogram for postoperative readmission and 3. To prospectively validate the readmission nomogram in an independent group of patients.

Methods

Patients who underwent general surgery procedures at a single institution were identified retrospectively from the prospectively maintained American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) database. ACS NSQIP data was paired with hospital billing data in order to identify readmissions which occurred within 30 days of discharge, as ACS NSQIP only reports readmissions within 30 days of surgery. This study was deemed minimal risk and was therefore declared exempt from IRB approval by the institutional IRB committee. Patients were included if they underwent an elective general surgery operation from 2006–2012. Exclusion criteria included death within 30 days of surgery and urgent or emergent operations. Emergent operations were defined by ACS NSQIP as patients who were deemed emergent by the attending surgeon and/or anesthesiologist. Patients with American Society of Anesthesiologists (ASA) classification of 5, pre-operative sepsis, dependence on ventilator at the time of surgery and pre-operative open wound were also excluded.

Explanatory variables included the following patient characteristics: age, gender, body mass index (BMI), ASA classification, functional status and 14 NSQIP defined pre-operative comorbidities. Age ≥ 65 was compared with <65 years of age. High risk BMI was defined as ≥ 30, while ASA classification was categorized as 1–2 versus 3–4. Functional status was evaluated as independent and dependent (partially and completely dependent) based on NSQIP definitions. NSQIP defined comorbidities included: diabetes mellitus, current smoker within one year of surgery, chronic obstructive pulmonary disease (COPD), ascites within 30 days prior to surgery, renal insufficiency, dialysis dependent, disseminated cancer, steroid use within 30 days of surgery, weight loss >10% in 6 months, pre-operative transfusion within 72 hours prior to surgery, bleeding disorder, hypertension, dyspnea and congestive heart failure (CHF) within 30 days of surgery. Intra-operative variables included procedure length, surgical specialty and wound class. Anatomic procedure types were assessed and as the study was not powered to analyze procedure specific risk for readmission, procedures were grouped into surgical specialties by the authors. Surgical specialties included advanced minimally invasive (MIS), colorectal, hepatopancreaticobiliary (HPB) and soft tissue oncology and general surgery. Wound class included clean, clean/contaminated, contaminated and dirty. Procedure length was evaluated as a dichotomous variable, with prolonged procedure length was defined as > 4^th quartile within each specialty (MIS >133 minutes, colorectal >259 minutes, general surgery >155 minutes, HPB >356 minutes). Post-operative variables included inhospital complications and discharge to higher level of care. In-hospital complications included any NSQIP defined complication which was diagnosed prior to discharge from the hospital. Patients were classified as being discharged to the same or higher level of care as compared with pre-admission level of care. The primary outcome of interest was readmission to the hospital within 30 days of discharge from the index hospitalization. Thirty day readmission data was extracted from the hospital billing database and linked with the institutional NSQIP database.

The study group consisted of patients who underwent an operation from 2006 – 2011. This group was used to develop a nomogram predictive of readmission. Then, over the course of 6 weeks (2013–2014), general surgery patients were identified from the daily operative schedule for a validation group, which was evaluated prospectively.

Descriptive statistics were performed to characterize the study populations. Chi square tests were used to evaluate for associations between explanatory variables and 30 day readmission in the study group. A correlation matrix was used to evaluate all explanatory variables for collinearity. Functional status and discharge to higher level of care were found to be highly correlated and were combined for multivariable analysis. A logistic regression analysis was then performed to identify variables predictive of readmission. In an effort to maximize the predictive ability of the model, all variables in the multivariable model were used to develop a prognostic nomogram using the linear predictor method.

We applied the nomogram to the study and validation populations to determine the nomogram predicted probability of readmission. A logistic regression analysis compared predicted (nomogram) probability of readmission with actual 30 day readmission in the validation group. The area under the curve was also calculated to quantify the accuracy of the nomogram. High risk for readmission was defined based on sensitivity and specificity from the area under the curve graph. The positive predictive value (PPV) and negative predictive value (NPV) were calculated for this readmission rate. All statistical analyses were performed in SPSS version 20. Significance was defined as p value <0.05.

Results

From our institutional NSQIP database, we identified 3,186 patients who underwent a general surgery procedure from 2006 – 2011. As demonstrated in Figure 1, after exclusions the study population consisted of 2,799 patients with a 10.2% readmission rate. The prospective validation population consisted of 255 patients, with 24 patients (9.4%) readmitted. The characteristics of both patient populations are listed in Table 1.

Figure 1.

Study and Validation Patient Populations The figure shows the original patient population, excluded patients, and the resulting patient population. ACS NSQIP indicates American College of Surgeons National Surgical Quality Improvement Program; UW, University of Wisconsin School of Medicine and Public Health billing data.

Table 1.

Characteristics of the Study Population^a

Characteristic	Study Population (n = 2799)	Prospective Validation Population (n = 255)
Age, median (range), y	56 (18–97)	54 (19–91)
≥65 y	812 (29.0)	72 (28.2)
Sex
Male	1208 (43.2)	133 (52.2)
Female	1591 (56.8)	122 (47.8)
Surgical specialty
Advanced MIS	343 (12.2)	30 (11.8)
Colorectal	1056 (37.7)	76 (29.8)
General	855 (30.5)	124 (48.6)
HPB and soft-tissue oncology	545 (19.5)	25 (9.8)
Complication in hospital	503 (18.0)	23 (9.0)

Abbreviations: HPB, hepatopancreaticobiliary; MIS, minimally invasive surgery.

^aValues are presented as number (percentage) of patients unless otherwise indicated.

We evaluated pre-operative, operative, and post-operative variables in association with 30 day readmission (Supplemental Table). Indicators of worse overall health including higher ASA class, dependent functional status, and recent weight loss, were associated with readmission. Colorectal and hepatopancreaticobiliary procedures were associated with readmission, as was longer operative time. In-hospital complications also correlated with readmission within 30 days from discharge.

Predictors of readmission are listed in Table 2. Bleeding disorder (OR2.549, 95% CI 1.464–4.440)), long operative time (OR 1.601, 95% CI 1.186–2.160) and in-hospital complications (OR 16.273, 95% CI 12.028–22.016), and dependent functional status or discharge to higher level of care (OR 1.937, 95% CI 1.176–3.190) all independently predicted 30 day readmission. Area under the curve was calculated with c statistic 0.797. The nomogram predicting readmission based on this linear regression model is shown in Figure 2. Risk for readmission can be determined by assigning points for each variable by drawing a line upward from the corresponding variable to the “Points” line, summing the points and identifying the prediction of 30 day readmission associated with the “Total Points” line.

Table 2.

Independent Predictors of Readmission

Variable	OR (95% CI)	P Value
Corticosteroids	1.453 (0.913–2.311)	.12
Weight loss	1.433 (0.836–2.455)	.19
Bleeding disorder	2.549 (1.464–4.440)	.001
Prolonged procedure length	1.601 (1.186–2.160)	.002
Specialtya
Colorectal	2.001 (1.111–3.601)	.02
General	1.536 (0.854–2.765)	.15
HPB and soft-tissue oncology	1.962 (1.076–3.576)	.03
ASA classification 3–4	1.177 (0.866–1.600)	.30
Wound classificationb
Clean contaminated	1.663 (1.091–2.535)	.02
Contaminated	1.620 (0.923–2.844)	.09
Dirty	2.426 (1.081–5.447)	.03
In-hospital complications	16.273 (12.028–22.016)	.002
Dependent functional status and/or higher care at discharge	1.937 (1.176–3.190)	.009

Abbreviations: ASA, American Society of Anesthesiologists; HPB, hepatopancreaticobiliary; OR, odds ratio.

^aBaseline variable: advanced minimally invasive surgery.

^bBaseline variable: clean wound.

Figure 2.

Nomogram to Predict Postoperative Readmission Within 30 Days After Hospital Discharge Points are assigned for each variable by drawing a line upward from the corresponding variable to the points line. The sum of the points plotted on the total points line corresponds with the prediction of 30-day readmission. ASA indicates American Society of Anesthesiologists; CR, colorectal; GS, general surgery; HPB, hepatopancreaticobiliary; MIS, minimally invasive surgery; N, no; and Y, yes. ^aProlonged operative time per surgical subspecialty: MIS, >133 minutes; GS, >155 minutes; HPB, >356 minutes; and CR, >259 minutes.

The nomogram was used to calculate predicted readmission risk for all patients. Readmission risk ranged from 3.6–53.9% (median 10.0%) in the study population and 3.6–32.7 % (median 8.89%) in the prospective validation population. Logistic regression and area under the curve were used to compare nomogram predictions with actual readmission rates. We found the nomogram to be quite predictive of risk for readmission in the prospective validation arm of the study with a c-statistic of 0.756 (OR 1.219, 95% CI 1.112–1.336).

Finally, in an attempt to assign a meaningful risk to patients using this new predictive tool, we investigated the negative and positive predictive value of a number of different risk estimates generated by the nomogram. We found that at a risk of readmission of 6%, the nomogram had a sensitivity and specificity for readmission prediction of 95% and 22% respectively. Using 6% as our cutoff for high risk, we calculated the positive and negative predictive values. The positive predictive value was 12.2% (study population) and 11.1% (validation population) and the negative predictive value was 97.6% (study population) and 97.9% (validation population).

Discussion

The aim of this study was to develop and validate a nomogram predictive of postoperative readmission in general surgery patients. Bleeding disorder, long operation, in-hospital complications, and dependent functional status or need for higher level of care at discharge independently predicted readmission within 30 days of discharge. Risk factors for readmission were used to produce a nomogram which was able to predict post-operative readmissions as validated with prospective analysis.

Other authors have identified similar risk factors for post-operative readmissions. Prolonged LOS and complications have been shown to be major drivers of readmission in general surgery patients.(3, 5, 6) Pre-operative comorbidity has previously been associated with surgical readmissions.(3, 5, 7) In a study of readmissions following major gastrointestinal resection, Kelly and colleagues also found prolonged operative time (>4 hours) to independently predict readmissions.(21) In order to evaluate risk for readmission in individual patients, a model is needed to incorporate these risk factors into a risk calculator.

A previous evaluation of multiple surgical subspecialties using national NSQIP data,(20) sought to develop a predictive model for post-operative readmissions. The authors identified prolonged LOS, in-hospital complications, and comorbidity as strong predictors of readmission. However, Lucas et al. found hepatopancreatobiliary patients to be at highest risk for readmission, while we identified colorectal patients to also be very high risk. This discrepancy is likely due to differing definitions of surgical subspecialties between the two studies. Other differences between this study and ours included the study populations; our study was limited to general surgery patients while Lucas and colleagues included vascular and thoracic surgery patients. Where national NSQIP readmission data is limited to 30 days from surgery, we were able to evaluate readmissions within 30 days from discharge by pairing institutional NSQIP data with hospital billing data.

The predictive ability of our model was acceptable (c statistic 0.756 in prospective validation), and was in line with the model created by Lucas et al(20) as well as many predictive models in the medical literature.(15) The high risk cutoff of 6% maximizes the sensitivity of the model in an effort to identify as many high risk patients as possible. Validation of our readmission nomogram yielded a high negative predictive value (98%). However, the positive predictive value was less impressive (<15%). Given that proposed interventions for high risk patients are low cost and low risk (post-discharge phone calls, early post-operative follow up, local lodging for those traveling from a distance), we utilized a cutoff that would identify most high risk patients. This model will allow providers to estimate a risk for readmission using readily available patient information at the time of discharge.

Currently, the discharge process is not standardized at our institution. Time to postoperative follow up, post-discharge phone calls, and other services are variable within and across surgical services. While medical patients at high risk for readmission are flagged in the electronic medical record, no such tool exists for surgical patients. This predictive nomogram will be used to implement quality improvement programs aimed at decreasing post-operative readmissions at our institution. We are using this nomogram to develop an online calculator and smart phone application so providers can easily calculate risk for readmission at the time of discharge. We plan to target patients at high risk for readmission for close outpatient follow up after discharge from the hospital, including nursing phone calls and earlier post-operative clinic visits. A systematic review of hospital readmissions in both medical and surgical patients demonstrated a 12–75% reduction in readmissions with the implementation of interventions aimed at decreasing readmissions in the majority of studies (14 of 19). Furthermore, of the 7 studies that evaluated mortality, 3 studies identified improved mortality after interventions were applied.(10) Naylor et al.(22) demonstrated an improvement in readmission rates with advanced practice nurse centered discharge planning and home follow up in elderly medical and cardiac surgery patients. Patients were given individual discharge plans, nursing phone follow up and home visits if needed. Patients in the intervention group incurred less costs and had longer time to readmission if they were readmitted.

This study has limited generalizability due to its inclusion of patients from a single institution. We were also unable to account for potential risk factors for readmission, such as social support, which were not recorded in our institutional NSQIP database. Ideally, we would have evaluated procedure specific readmission risk, however our study was not powered for this detailed analysis. To obtain adequate power for the study, we were unable to assess procedure specific risk for readmission. In the future, we plan to apply the nomogram to specific groups of patients in an effort to refine the risk calculator for various surgical subspecialties.

While the use of institutional NSQIP data may limit the applicability of our results to other institutions, in other ways it strengthened our study. We were able to pair hospital billing data with our NSQIP database in order to evaluate readmissions within 30 days of discharge as opposed to 30 days from surgery. This eliminates the immortal person time bias inherent to previously published readmission studies using NSQIP data. We were also able to perform chart review for all readmitted patients, which allowed us to exclude planned readmissions from our analysis. Another strength of this study is the prospective validation of the nomogram. A prospective validation more likely represents the population of patients who will be included in our future quality improvement projects using this tool and therefore we are confident that our model will be predictive of 30 day readmissions.

In summary, we have developed a nomogram predictive of readmission 30 days from discharge in general surgery patients. Prospective validation of the nomogram at a single institution demonstrated reasonable predictive ability. Application of this nomogram in the form of an online calculator at the time of discharge will better inform patients and providers of the risk for readmission. It will also allow for tailored discharge planning based on readmission risk.