20181088R1 EDITED BY DR SARR Simplified Risk Prediction Indices Do Not Accurately Predict 30-Day Death or Readmission After Discharge after Colorectal Surgery

Abstract

Background:

Risk-prediction indices are one of many tools implemented to guide efforts to decreasereadmissions. Butusing simplified models to predict a complex process can prove challenging. Additionally, no risk-prediction index has been developed for patients undergoing colorectal surgery. Therefore, we evaluated the performance of a widely-utilized simplified index (LACE) and a novel index in modeling readmissions in this patient population.

Methods:

Using a retrospective, split-sample cohort, patients discharged after colorectal surgery were identified within the inpatient databases of the Healthcare Cost and Utilization Project for New York, California, and Florida, 2006-2014. The primary outcome was death or readmission within 30 days after discharge. Multivariable logistic regression models incorporated patient comorbidities, postoperative complications, and hospitalization details, and were evaluated using the C statistic.

Results:

440,742 patients met eligibility criteria. The rate of death or readmission within 30 days after discharge was 14.0% (n = 61,757). When applied to surgical patients, the LACE index demonstrated a poor model fit (C = 0.631). Model fit improved significantly yet remained poor (C = 0.654; p < 0.001) with the addition of variables known to be associated with readmission after colorectal surgery: age, indication for surgery, and creation of a new ostomy. A novel, simplified model also yielded a poor model fit (C = 0.660).

Conclusion:

Post-discharge death or readmissionafter colorectal surgery is not accurately modeled using existing, modified, or novel simplified models of risk-prediction. Applying simplified models to complex processes such as colorectal surgery may not be appropriate, and payers and providers must ensure that efforts in quality improvement incorporate models that reflect the relevant patient population.

INTRODUCTION

Readmission rates have become a critical measure of quality for patients, physicians, hospitals, and payers. In a 2009 study, nearly 20% of over 11 million Medicare beneficiaries were readmitted within 30 days of discharge after an inpatient hospitalization; the cost of unplanned readmissions was estimated to exceed $17 billion dollars¹. Addressing readmission rates has quickly become a primary focus of efforts of quality improvement, in line with the goal of the Centers for Medicaid and Medicare Services (CMS) of improving the quality of health care while decreasing costs and also as well as pressure from public reporting and payers tying reimbursement to quality measures including readmissions^{2, 3}.

One common approach to identifying and addressing readmissions has been the development and incorporation of models of risk prediction. Patient variables are entered into a scoring system or index derived from a model of varying complexity and specificity to the patient population being studied. The score then allows providers to stratify patients by patient-specific risk, allowing for quality improvement teams to target resources such as pre- or post-discharge interventions to patients who are most likely to benefit^{4, 5}. One such readmission risk-prediction index, the LACE index composed of 4 variables during the 6 months prior to the index admission (Length of stay, Acute admission, Charlson Comorbidity Index score⁹, and the number of Emergency department (ED) visits), was developed at the hospital level from a combined medical and surgical patient population and has been validated and utilized across a number of specialties^6-8. The appeal of this index is its simplicity, because it uses only these four variables to model the predicted risk of death or unplanned readmission within 30 days after discharge¹⁰.

Patients undergoing colorectal surgery experience high readmission rates. In a recent, large, multi-institutional cohort, about 11% of such patients were readmitted, with a cost of approximately $9,000 per event¹¹. Despite this high financial burden, to date, no model of readmission risk-prediction or index specific to this patient population has been developed. Applying existing simplified models such as the LACE index may be attractive to quality improvement groups, but we hypothesized two major limitations to this approach. First, many models including the LACE index were developed at a hospital-level and may not be reflective of the risk factors unique to certain subsets of patients despite the aforementioned validation in other settings. Second, we believe that variables within the LACE model and therefore the model itself are more complex than advertised; tTherefore, this model may not be optimally pragmatic for implementation into accurate efforts of quality improvement in colorectal surgery. To address these potential limitations, we evaluated a spectrum of risk-prediction models for postoperative, post-discharge 30-day death or readmission for patients undergoing colorectal surgery. We started by evaluating the LACE index with the hypothesis that this model would require additional variables, presumably those with known associations with readmissions after intestinal surgery, to improve the model fit. Based on these results, we then developed a novel risk-prediction index in an attempt to maximize the model fit with the goal of devising a simple, risk-prediction index that could be employed pragmatically at the time of discharge.

MATERIALS AND METHODS

This study was approved as exempt from review by the Institutional Review Board at Washington University in St. Louis. Reporting follows guidelines of the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement for model development¹².

Data Source and Study Population

Patients 18 years and older who were discharged after operations of the the colon, rectum, or small intestine for any indication were identified from the State Inpatient Databases (SID) of the Agency for Healthcare Research and Quality (AHRQ) Healthcare Cost and Utilization Project (HCUP) for California (2006-2011), Florida (2006-2014), and New York (2006-2013)¹³. The included operations were defined using procedure codes from the International Classification of Diseases, 9^th Revision, Clinical Modification (ICD-9-CM; Table 1). To allow for adequate post-discharge follow-up, patients were ineligible if they were not a resident of the state in which they were admitted or if they were admitted in the final quarter of the final year of sourced SID data. Records with overlapping admission and discharge dates were collapsed into a single admission. Patients were excluded if their index postoperative length of stay was less than two days or greater than 21 days; lesser durations of hospitalization were assumed to represent coding errors, as the expected length of stay for all included procedures would be at least two days, and patients staying greater than 21 days were assumed to have a severity of illness that would not inform a generalizable model of risk-prediction. Patients discharged to hospice were excluded.

TABLE 1. ICD-9 procedure codes for included operationsof the colon, rectum, and small bowel.

Any cases involving small or large intestine resections for trauma were excluded (45.61, 45.71).

Procedure	ICD-9 Procedure Codes
Large Intestine
Colectomy, partial or total, including laparoscopic	17.31/2/3/4/5/6/9; 45.72/3/4/5/6/9; 45.81/2/3
Other, including exteriorization, revision, or closure of stoma*	46.04; 46.40/3; 46.50/2
Colostomy	46.03; 46.10/1/3/4
Rectal
Abdominoperineal resection	48.50/1/2/9
Other	48.61/2/3/4/5/9
Small Bowel
Ileostomy	46.01; 46.20/1/2/3/4
Other, including ileal pouch or closure of stomaa	45.62, 45.95, 46.02, 46.51

^aStoma closures (46.50/1/2) were included only if billed with another procedure on this list.

Follow-up occurred up to the earliest of 30 days after discharge, date of discharge to hospice, or date of death. The primary outcome was death or readmission within 30 days after discharge. Readmission was defined as any inpatient admission of any duration to any hospital within the SID afterdischarge from the index surgical hospitalization. Readmissions were omitted if they involved codes for trauma (Table S1). Post-index discharge death was identified if it occurred during a SID or if it was recorded in an emergency department visit using the HCUP State Emergency Department Databases (SEDD).

Statistical Analysis

Between-group comparisons were made using chi-square tests or Wilcoxon-Mann-Whitney tests. Multivariable logistic regression was used to model for death or readmission within 30 days after discharge after intestinal surgery. Model fit was calculated using the true C statistic for non-hierarchical logistic regression models to allow for comparison with the methods of the original LACE manuscript. Hierarchical modeling at the hospital level was used for the novel model and model fit was therefore, calculated using an estimated C statistic¹⁴. Goodness-of-fit was assessed using the Hosmer-Lemeshow test for non-hierarchical logistic regression.

Variable selection was based on previously published and hypothesized associations between the variables and readmission after discharge aftercolorectal surgery (Table S1). Variables with known associations included age, new colostomy or ileostomy, length of stay, comorbidity, discharge destination, and surgical site infection^{1, 11, 15}. Other variables tested included the following: indication for surgery (coded hierarchically and exclusively starting with malignancy, followed by diverticulitis, inflammatory bowel disease, and all others), variables for postoperative complications, select AHRQ Patient Safety Indicators¹⁶, and select HCUP comorbidity and complication diagnoses from the Clinical Classifications Software¹⁷. Comorbidities were defined using the standardized ICD-9 codes for the Elixhauser Comorbidity Index; although the Charlson Comorbidity Index was used in the LACE model; the Elixhauser Comorbidity Index has shown greater discrimination when applied to administrative data^{9, 18, 19}.

Variables were added sequentially based on (1) association with the outcome of p<0.001 on univariate logistic regression and (2) pragmatism based on perceived ease of assessing and scoring at the bedside at the time of discharge. Variables including interactions were not incorporated into the model if they did not improve the C statistic by at least 0.001. Once the C statistic was maximized, a scoring index was developed following the methods of Sullivan et al²⁰. Because a substantial body of literature is known about the risk of readmissions associated with colorectal surgery for cancer^{21, 22}, a baseline point value of 1 was assigned to the parameter estimate for the risk associated with cancer as the indication for surgery. The expected probability was calculated using the same method as van Walraven et al, and confidence intervals for observed probabilities were determined using the exact method^{10, 23}. All analyses were performed in SAS, version 9.4 and SAS Enterprise Guide version 7.1 (SAS Institute, Cary, NC).

RESULTS

440,742 eligible patients underwent intestinal surgery in 795 hospitals across three states between 2006 and 2014 (Figure 1). The procedures included for colon, small bowel, and rectal procedures are listed in Table 1. The most frequent indication for operationwas primary or secondary malignancy (41.4%, n = 182,412). Demographic and clinical variables are reviewed in Table 2.

Figure 1. Inclusion scheme.

^aSee Methods section for state-specific inclusion periods

TABLE 2.

Demographic and clinical variables for the index surgical admission, stratified by death or readmission within 30 days after discharge afterintestinal surgery.

	Total Cohort		Readmission or Death Within 30 Days		No Event		Univariate p Value
	n or median	% or IQR	n or median	% or IQR	n or median	% or IQR
n	440,742		61,757	14.0%	378,985	86.0%
Age	65 years	53 - 76	66	54 - 77	65	53 - 75	<0.001
Female	240,945	54.7%	34,246	55.5%	206,699	54.5%	0.001
Charlson Comorbidity Index Score	2	0 - 3	2	0 - 5	2	0 - 3	<0.001
Race/Ethnicity							<0.001
White	316,113	71.7%	43,882	71.1%	272,231	71.8%
Black	36,528	8.3%	5,955	9.6%	30,573	8.1%
Hispanic	51,416	11.7%	7,606	12.3%	43,810	11.6%
Other	27,012	6.1%	3,593	5.8%	23,419	6.2%
Missing	9,673	2.2%	721	1.2%	8,952	2.4%
Primary Payer							<0.001
Private	167,389	38.0%	19,317	31.3%	148,072	39.1%
Medicare	219,817	49.9%	34,124	55.3%	185,693	49.0%
Medicaid	29,537	6.7%	5,219	8.5%	24,318	6.4%
Other	23,999	5.4%	3,097	5.0%	20,902	5.5%
Procedurea
Colon	286,157	64.9%	39,831	64.5%	246,326	65.0%	0.016
Small Intestine	107,496	24.4%	21,074	34.1%	86,422	22.8%	<0.001
Rectal	56,861	12.9%	8,780	14.2%	48,081	12.7%	<0.001
Includes Colostomy	55,929	12.7%	10,199	16.5%	45,730	12.1%	<0.001
Includes Ileostomy	27,015	6.1%	7,523	12.2%	19,492	5.1%	<0.001
Indication							<0.001
Malignancy	182,412	41.4%	22,544	36.5%	159,868	42.2%
Diverticulitis	86,468	19.6%	2,376	3.8%	11,109	2.9%
Inflammatory Bowel Disease	13,485	3.1%	27,666	44.8%	130,711	34.5%
Other	158,377	35.9%	9,171	14.9%	77,297	20.4%
Emergent Admission	161,118	36.6%	27,278	44.2%	133,840	35.3%	<0.001
Length of Stay	7 days	5 - 10	8	6 - 13	7	5 - 10	<0.001
Discharge Destination							<0.001
Home	282,444	64.1%	30,509	49.4%	251,935	66.5%
Home with Home Health	104,695	23.8%	18,703	30.3%	85,992	22.7%
Other Facility	52,651	11.9%	12,328	20.0%	40,323	10.6%
Other	952	0.2%	217	0.4%	735	0.2%

^aPercentages are greater than 100%, as the coding allowed for multiple procedure types to occur in a single case

61,757 patients (14.0%) experienced the primary outcome of death or readmission within 30 days after postoperative discharge. Death made up a small amount of these events (n = 1,806; 2.6% of all patients experiencing the outcome; 0.4% of all discharged patients). The median number of days from discharge to death or readmission was 8 days (interquartile range (IQR) 4 – 16 days). Patients experiencing death or readmission were older and had a greater postoperative length of stay among other significant differences (Table 2).

LACE Evaluation

To evaluate the performance of the LACE model, a split-sample cohort was created randomly with separate derivation and validation populations (n = 220,371 for each). The rate of death or readmission within 30 days after discharge was not different between the groups (14.0% for each; p = 0.494).

The C statistic of the LACE model in its original publication was 0.700, which was derived at the hospital level from a combined cohort of medical and surgical patients¹⁰. In the colorectal derivation cohort, all four LACE variables were significantly associated with 30-day readmission or death (Table 3); however, the C statistic of the LACE index in this derivation cohort was poor at 0.631. The calibration curve of this model in our derivation cohort demonstrated that the LACE index underestimated the risk of death or readmission by approximately 5% for the majority of patients (Figure 2A).

TABLE 3. Comparison of LACE model and a model with additional variables specific to readmissions after colorectal surgery.

Three of the four original LACE variables were not as strongly associated with death or readmission within 30 days after discharge for patients undergoing intestinal surgery as they were for the original LACE population (van Walraven et al., 2010). The addition of variables specific to readmissions after colorectal surgery improved the model fit (C statistic increased from 0.631 to 0.654).

	Intestinal Surgery, Derivation Cohort		Original LACE Cohortb
	Odds Ratio	95% CI	Odds Ratio	95% CI
Original LACE Variables
Length of Staya	1.87	1.82 - 1.92	1.47	1.25 - 1.73
Acute Admission	1.07	1.04 - 1.10	1.84	1.29-2.63
Charlson Comorbidity Index Score	1.08	1.07 - 1.08	1.21	1.10 - 1.33
Visits to Emergency Department During Previous 6 Monthsa	1.34	1.31 - 1.37	1.56	1.27 - 1.92
	C statistic = 0.631		C statistic = 0.700
Modified LACE, Adding Variables Specific to Colorectal Surgery
Length of Stay	1.66	1.61 - 1.70
Acute Admission	0.97	0.94 – 0.996
Charlson Comorbidity Index Score	1.09	1.08 - 1.10
Visits to Emergency Department During Previous 6 Months	1.32	1.29 - 1.35
Age (Five-Year Increments)	1.02	1.01 - 1.02
Indication for Surgery
Diverticulitis	1.00	-
Malignancy	0.95	0.92 – 0.99
Inflammatory Bowel Disease	1.44	1.34 - 1.56
Other	1.49	1.44 - 1.53
Colostomy creation	1.22	1.18 - 1.27
Ileostomy creation	2.18	2.09 - 2.27
	C statistic = 0.654

^aRetained original transformation of continuous variables from van Walraven et al¹⁰

^bData from the original medical and surgical population from van Walraven et al¹⁰

Figure 2. Calibration curves for death or readmission within 30 days after discharge afterintestinal surgery.

(A) The original LACE index (van Walraven et al., 2010), when applied to the intestinal surgery derivation population, underestimates the likelihood of death or readmission for the majority of the possible scores. (B) The COILED index demonstrates improved fit of observed versus expected incidence of the outcome when compared to the LACE model.

Next, we evaluated whether the addition of variables with known associations between colorectal surgery and postoperative readmissions could improve the performance of the LACE model. These variables included age, indication for operation, and creation of a new ostomy. The addition of these variables to the LACE model increased the model discrimination, but the model fit remained poor (C = 0.654 from 0.631; p < 0.001; Table 3).

Finally, after observing that increasing the complexity through a modified LACE model with eight variables resulted in a complex yet poor model, we then evaluated whether a new, simple, and pragmatic model could achieve equal or greater discrimination. The primary aim of this model would be pragmatism, i.e. the model should be able to be applied at the bedside at the time of discharge without relying on complex systems of scoring or searching a patient’s history via the medical record. After sequential model building (all tested variables are listed in Table S1), a final model called the COILED index, was derived that incorporated six variables (Table 4): Comorbidity (limited to chronic lung disease or heart failure, as defined by the ICD-9 codes from the Elixhauser comorbidity measures¹⁹), Ostomy created at the index surgery, Indication for surgery, Length of stay, _ED visits within the 6 months prior to surgical admission, and Discharge Destination. Notably, postoperative complications did not significantly impact the model fit.

TABLE 4.

COILED index (19) for the risk of death or readmission within 30 days after discharge after intestinal surgery.

	Variable	Points	Odds Ratio	95% CI
C	Comorbidity: Chronic Pulmonary Disease or Heart Failurea	1	1.29	1.25 - 1.33
O	Ostomy Creation
	Colostomy	1	1.15	1.11 - 1.19
	Ileostomy	3	1.99	1.90 - 2.08
I	Indication for Surgery
	Diverticulitis	0	1.00	-
	Cancer	1	1.23	1.19 - 1.28
	Inflammatory Bowel Disease	1	1.33	1.24 - 1.44
	Other	2	1.51	1.46 - 1.57
L	Length of Stay (Postoperative)		1.56	1.52 - 1.60
	2 - 3 days	0
	4 - 5 days	1
	6 - 7 days	2
	8 - 14 days	3
	15 - 21 days	4
E	ER visits in the 6 months prior to the index surgical admission	1	1.33	1.29 - 1.37
D	Discharge Destination
	Home	0	1.00	-
	Home With Home Health	1	1.27	1.23 - 1.32
	Facility: Skilled Nursing, Rehab, or Other	2	1.63	1.56 - 1.70
TOTAL POSSIBLE POINTS		14

^aFrom the Elixhauser ICD-9 coding system¹⁹. Chronic pulmonary disease includes diagnoses such as chronic obstructive pulmonary disease (ICD-9 code 491.20), emphysema (492.8), and asthma (493)

Although the model discrimination was improved compared to the original LACE index (C = 0.652; Figure 2B), this model still poorly fit the data as confirmed by the Hosmer-Lemeshow goodness-of-fit statistic (p < 0.001). The C statistic of the COILED model in the validation cohort was 0.654 and for the entire cohort was 0.653. Hierarchical modeling at the hospital level resulted in an estimated C statistic of 0.660.

DISCUSSION

The use of indices of risk-prediction in an effort to decreasereadmissions has become a rapidly expanding area of study in quality improvement and implementation research. Of course, the primary driver of these efforts is to improve the outcomes of our patients, but the impact of external pressure from payers to tie reimbursement to quality measures such as readmissions as well as the public reporting of these measures cannot be denied. Simplified models such as the LACE index are attractive in that they potentially offer high-yield information with relative ease-of-use. When our institution identified these advantages and sought to apply the LACE model to decrease high readmission rates for patients undergoing intestinal surgery, we challenged the appropriateness of this plan, knowing that LACE was developed at the hospital level from a cohort containing few patients undergoing colorectal surgery; accordingly, many variables known to be associated with readmission after intestinal surgery were not represented in LACE. Therefore, we evaluated the performance of the LACE model in this specific patient population of patients undergoing intestinal surgery and determined that it was a poor predictor of readmissions. Enhancement of the LACE model by adding variables known to be associated with readmission after intestinal surgery did not improve the model fit. Finally, we created a novel model designed for ease-of-application at the bedside at the time of discharge, but this also resulted in a poor model fit. We concluded that in this large, multi-state, administrative database of patients undergoing colorectal surgery, we could not reliably predict readmissions after intestinal surgery using these simplified models.

Acknowledging the limitations of simplified readmissions models has been reported frequently. A 2011 systematic review of models of readmissions risk-prediction determined that many indices for the prediction of readmission-risk performed poorly, and “efforts to improve their performance are needed as [their] use becomes more widespread”⁵. In this same systematic review, the LACE index was highlighted as having good performance for a four-variable model and, since, has been cited widely and modified ^{6-8, 24, 25}. Our institution identified the LACE index for possible implementation in our efforts at quality improvement, but we had concerns regarding two perceived limitations of the LACE model.

• (ELSEVIER I AM NOT CERTAIN IF WE SHOULD leAVE THIS INDENTED CAN YOU DECIDE THANKS) The first limitation was that the variables we perceived to be major contributors to readmissions after colorectal surgery were not represented in the LACE index. This determination led us to a logical but perhaps often overlooked conclusion that model performance cannot be assumed when intending to apply a model to a different patient population. Although the LACE index had reasonably good performance (C = 0.700), it was developed at the hospital level using a combined medical and surgical patient population representing a large diversity of principal diagnoses and procedures, of which very few were intestinal operations; indeed, only 1.7% (n=81) of the patients in the LACE dataset underwent colorectal surgery, and of these, only five (6.2%) experienced postoperative death or readmission, far below the published rates of death or readmission after colorectal surgery of 10-17%^{11, 21, 22, 26}. In our study, we demonstrated that this model does not perform well when applied to patients undergoing colorectal surgery.

• The second limitation we addressed was that we suspected that the LACE model when incorporated is more complex than advertised. An ideal simple model in our opinion should be pragmatic, designed for ease-of-use at the time of discharge by minimizing the complexity of variables and not requiring access to databases or the electronic medical record (EMR). Within the original LACE index, the Charlson Comorbidity Index is considered a single variable, yet the scoring system is quite complex, with 19 diagnoses scored on a tiered system⁹. Without access to an EMR, reliable coding, or recall of a patient’s comorbidities, the Charlson Comorbidity Index would be challenging to implement efficiently at the time of discharge. We addressed this limitation by evaluating single comorbidities, particularly those known to be associated with high rates of readmissions, such as chronic cardiac or pulmonary conditions, as well as interaction variables¹. A second hidden complexity of the LACE model is the inherent challenge of recall bias. Using the number of visits to an emergency room in the 6 months prior to the index admission would be difficult to assess based on patient recall without the aid of an EMR or large administrative datasets like the one used here. To address this limitation, we decreasedthis original LACE variable to a binary variable for any number of emergency room visits to simplify assessment and decrease error by recall bias^{27, 28}.

The impact of this work can be far-reaching. Locally, after sharing our findings, the implementation of LACE into our EMR and pro esses of discharge screening have been stopped, impacting our 1,400 annual, colorectal postoperative admissions. Other surgical subspecialties searching for strategies to decrease readmission rates have similarly declined the use of of simplified models. We suspect that the maximal effect of interventions targeting the reduction of readmisiions will require models designed to predict readmission risk that are specific to both the relevant patient population and the setting of delivery of care, be it specific geographic regions, hospitals, or provider. Efforts are underway at our institution to examine these and other models. We hope the findings presented here and the subsequent actions taken at our institution can be generalized to others, and further hope our findings can result in action on a larger scale. Although intestinal surgery is not covered in the current formula for calculating the Payment Adjustment Factor for the Readmissions Reduction Program³, it is easy to envision a future in which such efforts to tie performance to reimbursement will extend to many aspects of our practices within our specialty. Surgeons must be vigilant and active participants in the derivation and implementation of these quality measures and formulas. To that end, our results have been shared with the Quality Assurance and Safety Committee of the American College of Colon and Rectal Surgeons. From committees like this one and through further dissemination, we hope that these data can be used to shape policy by demonstrating that the application of generalized models to specific patient populations may not be effective and that simplified models may not predict complex processes accurately such as postoperative readmissions. Enforced application of imprecise models or formulas, particularly through rewards or penalties as crucial as reimbursement, may mislead the efforts of quality improvement specialists and may not result in benefit to patients.

There are a number of limitations to our work. First, we used both objective and subjective criteria to select variables. Our hypothesis was to evaluate simple and pragmatic models, so we limited our study to those that could be evaluated easily at the bedside at the time of discharge, a definition for which there is no singular objective criteria. Second, we anticipate criticism regarding our finding that complications did not affect a model predicting readmissions. We thoroughly evaluated complications by incorporating validated complication coding systems^{17, 19}, nationally-standardized quality measures¹⁶, and an extensive novel list of ICD-9 diagnoses. Although we assume that certain complications are critical in assessing readmission risk, we assume that the effect size of single complications or larger categories of complications are quite small in such a large database. Additionally, the coding of complications specific to colorectal surgery in an administrative database may not be as robust as the coding of complications within a quality improvement database, and similarly, the severity of these complications cannot be elucidated easily in an administrative database.. Additional limitations include the unknown rate of post-discharge mortality if it occurred as an outpatient and not evaluating the indication for readmission, which is not easily accomplished with administrative data²⁹. Finally, patient socioeconomic variables known to be important when considering readmission rates and targeting interventions to patients to reduce post-discharge readmissions^{30, 31} were not evaluated fully in this model, because they are not well-represented in administrative billing data.

CONCLUSION

Using hospital-based data, we determined that 30-day readmission or death afterdischarge after intestinal surgery is not well-modeled using existing, modified, or novel, simple, pragmatic indices ofor prediction of readmission. Providers must evaluate simplified models carefully when planning and implementing efforts of quality improvement to address complex issues such as postoperative readmissions and must be active participants in ensuring appropriate derivation and application of such models at the local and national level.