Abstract
Background
Readmissions to hospital are common, costly and often preventable. An easy-to-use index to quantify the risk of readmission or death after discharge from hospital would help clinicians identify patients who might benefit from more intensive post-discharge care. We sought to derive and validate an index to predict the risk of death or unplanned readmission within 30 days after discharge from hospital to the community.
Methods
In a prospective cohort study, 48 patient-level and admission-level variables were collected for 4812 medical and surgical patients who were discharged to the community from 11 hospitals in Ontario. We used a split-sample design to derive and validate an index to predict the risk of death or nonelective readmission within 30 days after discharge. This index was externally validated using administrative data in a random selection of 1 000 000 Ontarians discharged from hospital between 2004 and 2008.
Results
Of the 4812 participating patients, 385 (8.0%) died or were readmitted on an unplanned basis within 30 days after discharge. Variables independently associated with this outcome (from which we derived the nmemonic “LACE”) included length of stay (“L”); acuity of the admission (“A”); comorbidity of the patient (measured with the Charlson comorbidity index score) (“C”); and emergency department use (measured as the number of visits in the six months before admission) (“E”). Scores using the LACE index ranged from 0 (2.0% expected risk of death or urgent readmission within 30 days) to 19 (43.7% expected risk). The LACE index was discriminative (C statistic 0.684) and very accurate (Hosmer–Lemeshow goodness-of-fit statistic 14.1, p = 0.59) at predicting outcome risk.
Interpretation
The LACE index can be used to quantify risk of death or unplanned readmission within 30 days after discharge from hospital. This index can be used with both primary and administrative data. Further research is required to determine whether such quantification changes patient care or outcomes.
Readmission to hospital and death are adverse patient outcomes that are serious, common and costly.1,2 Several studies suggest that focused care after discharge can improve post-discharge outcomes.3–7 Being able to accurately predict the risk of poor outcomes after hospital discharge would allow health care workers to focus post-discharge interventions on patients who are at highest risk of poor post-discharge outcomes. Further, policy-makers have expressed interest in either penalizing hospitals with relatively high rates of readmission or rewarding hospitals with relatively low expected rates.8 To implement this approach, a validated method of standardizing readmission rates is needed.9
Two validated models for predicting risk of readmission after hospital discharge have been published.10,11 However, these models are impractical to clinicians. Both require area-level information (e.g., neighbourhood socio-economic status and community-specific rates of admission) that is not readily available. Getting this information requires access to detailed tables, thereby making the model impractical. Second, both models are so complex that risk estimates cannot be attained from them without the aid of special software. Although these models have been used by health-system planners in the United Kingdom, we are unaware of any clinicians who use them when preparing patients for hospital discharge.
Our primary objective was to derive and validate a clinically useful index to quantify the risk of early death or unplanned readmission among patients discharged from hospital to the community.
Methods
Study design
We performed a secondary analysis of a multicentre prospective cohort study conducted between October 2002 and July 2006. The study involved patients discharged to the community from the medical or surgical services of 11 hospitals (6 university-affiliated, 5 community) in five cities in Ontario after an elective or emergent hospital admission. To be eligible for inclusion, patients had to be adults, provide informed consent, have a telephone (to participate in follow-up telephone interviews), and be cognitively intact (to ensure validity of the consent process and accuracy of information given in the interviews). We recruited patients of medical and surgical services because such patients comprise most discharges from hospitals.
Given that the process of health care provided in nursing homes differs from processes of care for patients in the community (i.e., nursing home residents frequently receive care from an onsite physician), we restricted eligibility to patients who were not residents of nursing homes.
Before discharge from hospital, patients were interviewed by study personnel to identify their baseline functional status, living conditions and chronic medical conditions. Chronic medical conditions were confirmed by a review of the patient’s chart and hospital discharge summary, when available. The chart and discharge summary were also used to identify diagnoses made while in hospital and medications given at discharge. All medications given at discharge were compared with those documented on the admission note to determine which discharge medications had been started in hospital.
To determine whether patients had had an unplanned readmission to hospital or had died within 30 days of discharge, we contacted either patients or their principal contacts (identified by each patient at recruitment) one month after discharge. We combined unplanned readmissions with deaths to avoid bias caused by censoring deaths when hospital readmission alone is examined.12 We classified readmissions as unplanned if they had not been arranged or planned when the patient was originally discharged from hospital.
We chose a 30-day time frame for our primary outcome to increase the likelihood that poor outcomes would be related to the index admission or discharge process and would be more likely to be remediable. When analyzing this outcome by its components, we classified patients who were urgently readmitted within 30 days as having died if they subsequently died within 30 days of discharge from hospital.
Our study was approved by the Ottawa Hospital Research Ethics Board and by each participating site.
Index derivation and internal validation
We randomly selected half of the participants for index derivation and used the other half for internal validation. We identified all patient-level and admission-level variables among the data set that we thought might influence outcomes. We used multivariable logistic regression to measure the independent association of these factors with early death or unplanned readmission to hospital. Patient-level variables and admission-level variables entered into the model are listed in Table 1. We also offered the presence or absence of most common diagnoses and procedures (Appendix 1, available at www.cmaj.ca/cgi/content/full/cmaj.091117/DC1).
Table 1.
Characteristics of 4812 participants discharged from hospital to the community, by outcome within 30 days after discharge
| Characteristic | Overall no. (%) of patients*n = 4812 | Death or unplanned readmission within 30 days, no. (%) of patients* | |
|---|---|---|---|
| No n = 4427 (92.0) | Yes n = 385 (8.0) | ||
| Patient variable | |||
| Age at index admission, yr, mean (SD) | 61.3 (17.0) | 61.0 (17.0) | 64.7 (16.5) |
| Female | 2530 (52.6) | 2323 (52.5) | 207 (53.8) |
| Living alone | 1127 (23.4) | 1033 (23.3) | 94 (24.4) |
| Dependent for one or more ADL | 323 (6.7) | 280 (6.3) | 43 (11.2) |
| Charlson comorbidity index score† | |||
| Median (IQR) | 0 (0–0) | 0 (0–0) | 0 (0–2) |
| > 0 | 1128 (23.4) | 984 (22.2) | 144 (37.4) |
| Hospital admissions during previous 6 mo | |||
| Median (IQR) | 0 (0–1) | 0 (0–1) | 0 (0–1) |
| > 0 | 1557 (32.4) | 1375 (31.1) | 182 (47.3) |
| Visits to emergency department during previous 6 mo | |||
| Median (IQR) | 0 (0–1) | 0 (0–1) | 1 (0–2) |
| > 0 | 1750 (36.4) | 1543 (34.8) | 207 (53.8) |
| Has regular physician | 4580 (95.2) | 4208 (95.1) | 372 (96.6) |
| Admission variable | |||
| Medical care | 2160 (44.9) | 1922 (43.4) | 238 (61.8) |
| Emergent admission | 2796 (58.1) | 2505 (56.6) | 291 (75.6) |
| Emergent surgery during admission | 391 (8.1) | 367 (8.3) | 24 (6.2) |
| Length of stay, d, median (IQR) | 5 (2–8) | 4 (2–8) | 7 (4–12) |
| Medications at discharge, no., median (IQR) | 4 (2–7) | 4 (2–7) | 5 (3–8) |
| New medications at discharge | |||
| Median (IQR) | 0 (0–2) | 0 (0–2) | 1 (0–2) |
| > 0 | 2340 (48.6) | 2139 (48.3) | 201 (52.2) |
| Season at discharge | |||
| Spring | 1722 (35.8) | 1665 (35.8) | 57 (34.7) |
| Summer | 1008 (21.0) | 984 (21.2) | 24 (14.6) |
| Autumn | 830 (17.3) | 785 (16.9) | 45 (27.4) |
| Winter | 1252 (26.0) | 1214 (26.1) | 38 (23.2) |
| Consultations in hospital, median (IQR) | |||
| Median (IQR) | 0 (0–1) | 0 (0–1) | 1 (0–1) |
| > 0 | 1848 (38.4) | 1654 (37.4) | 194 (50.4) |
| Complications while in hospital, median (IQR) | |||
| Median (IQR) | 0 (0–0) | 0 (0–0) | 0 (0–0) |
| > 0 | 615 (12.8) | 542 (12.2) | 73 (19.0) |
We used fractional polynomial functions to determine the best linear or nonlinear form for continuous variables.15–17 Backward stepping with an α-error criterion of 0.01 was used to include only significant variables in the final multivariable logistic model. We used an inclusion criterion of 1% to ensure model stability. To account for clustering of patients within hospitals, we used generalized estimating equation methods. We found no changes in the parameter estimates for all model variables, and all variables remained significant at the 5% level.
We then used the methods described by Sullivan and colleagues18 to modify the final logistic model into a risk index. The number of points assigned to each significant covariate equaled its regression coefficient divided by the parameter estimate in the model with the smallest absolute value rounded to the nearest whole number. We then calculated each participant’s final score by summing up his or her points. The expected probability of early death or unplanned readmission associated with each score was the inverse of 1 + e–(intercept+b*total score), where b was the value of the coefficient in the regression model with the smallest absolute value.
External validation
Since the components of the final model were available from administrative data, the index was externally validated using three population-based administrative databases that capture data on all Ontarians. The Discharge Abstract Database records all hospital admissions. The National Ambulatory Care Reporting System records all emergency department visits, and the Registered Patient Database records all dates of death.
We used the Discharge Abstract Database to randomly select 1 000 000 (of 1 916 398) adult medical or surgical patients out of discharged to the community from Ontario hospitals between April 2004 and January 2008. This period was used to ensure that six months of preadmission data existed in the National Ambulatory Care Reporting System for all participants. The urgency and length of stay of each hospital admission were noted.
We calculated each patient’s Charlson comorbidity index score using the International Classification of Disease (ICD) codes cited by Quan and colleagues.19 We linked to the National Ambulatory Care Reporting System to measure the number of visits to an emergency department by each patient in the six months before admission. We determined patients’ status at 30 days post-discharge by linking to the Registered Patient Database for data related to deaths and to the Discharge Abstract Database for data related to unplanned urgent readmissions.
Assessment of risk score
We used a C statistic with 95% confidence intervals (CIs)20 to measure the ability of the index to discriminate between patients who died or had an unplanned readmission within 30 days of discharge and those who did not. The C statistic expresses the proportion of times that the case in each case–noncase pair has a higher model-based predicted risk of the outcome. We measured the calibration of the score by comparing the observed and expected numbers of patients with the outcome for each score. We deemed the observed and expected death or urgent readmission rates to be similar if the 95% CI around the observed rate included the expected rate. We calculated 95% CIs for observed rate of death or urgent readmission rates using exact methods.21 We summarized overall calibration using a Hosmer–Lemeshow goodness-of-fit test.22
Results
Between October 2002 and July 2006, we enrolled 5035 patients from 11 hospitals. We determined outcome status at 30 days for 4812 patients (95.6%). Of the remaining patients, 124 (2.5%) refused participation when contacted for follow-up, 83 (1.6%) were lost to follow-up, and 16 (0.3%) were removed from the study because they were admitted to a nursing home during the first month after discharge from hospital.
The study cohort is described in Table 1. Participants were middle-aged, and almost 95% were independent with regard to activities of daily living. Most participants were free of serious comorbidities, with more than 75% having a Charlson comorbidity index score of zero.13 Most admissions were emergent (58.1%), and almost half (44.9%) were to a medical service. The most common reasons for hospital admission included acute coronary syndromes, cancer diagnosis and complications, and heart failure (Appendix 1). Coronary artery bypass grafting and arthroplasty were the most common procedures. Patients in the derivation (n = 2393) and validation (n = 2419) cohorts were similar.
During the first 30 days after discharge, 385 (8.0%) patients died or were urgently readmitted (death 36 [9.4% of outcomes], unplanned readmission 349 [90.6% of outcomes]). Patients with one of the primary outcomes had more emergency department visits before admission and were more likely to be admitted emergently and for longer durations than patients who did not die (Table 1). Most other patient-related and admission-related variables appeared to have little influence on risk of early death or unplanned readmission.
Index derivation and internal validation
Only four variables were independently associated with death or readmission within 30 days after discharge (Table 2). These variables were length of stay (“L;” odds ratio [OR] 1.47, 95% CI 1.25–1.73) acuity of the admission (“A;” odds ratio [OR] 1.84, 95% CI 1.29–2.63), patient comorbidity (as measured using the total Charlson comorbidity index score) (“C;” odds ratio [OR] 1.21, 95% CI 1.10–1.33), and emergency department use (measured as the number of visits in the previous six months) (“E;” odds ratio [OR] 1.56, 95% CI 1.27–1.92). Length of stay was modelled as a logarithm and the number of emergency department visits was modelled as a square root term. We found no significant interactions between these or other variables. The final logistic model was moderately discriminative (C statistic 0.700) and was well calibrated (Hosmer–Lemeshow goodness of fit statistic 6.99, 8 degrees of freedom, p = 0.54). None of the other variables listed in Table 1 met our criteria for inclusion in the model.
Table 2.
Final logistic regression model for risk of death or unplanned readmission within 30 days after discharge (derivation group only, n = 2393)
| Variable | Odds ratio (95% CI) |
|---|---|
| Length of stay in days (logarithm) | 1.47 (1.25–1.73) |
| Acute (emergent) admission | 1.84 (1.29–2.63) |
| Comorbidity (Charlson comorbidity index score) | 1.21 (1.10–1.33) |
| Visits to emergency department during previous 6 mo, (√) | 1.56 (1.27–1.92) |
Note: CI = confidence interval.
We modified this logistic model into an index to predict early death or unplanned readmission (Table 3). To facilitate recall of the components of the index, we titled the index using a simple mnemonic. The LACE index had a potential score ranging from 0 to 19. The total LACE score in the study population had a normal distribution that was slightly skewed to the right (Figure 1).
Table 3.
LACE index for the quantification of risk of death or unplanned readmission within 30 days after discharge
| Attribute | Value | Points* |
|---|---|---|
| Length of stay, d (“L”) | < 1 | 0 |
| 1 | 1 | |
| 2 | 2 | |
| 3 | 3 | |
| 4–6 | 4 | |
| 7–13 | 5 | |
| ≥ 14 | 7 | |
| Acute (emergent) admission (“A”) | Yes | 3 |
| Comorbidity (Charlson comorbidity index score†) (“C”) | 0 | 0 |
| 1 | 1 | |
| 2 | 2 | |
| 3 | 3 | |
| ≥ 4 | 5 | |
| Visits to emergency department during previous 6 mo (“E”) | 0 | 0 |
| 1 | 1 | |
| 2 | 2 | |
| 3 | 3 | |
| ≥ 4 | 4 |
A patient’s final LACE score is calculated by summing the points of the attributes that apply to the patient.
The Charlson comorbidity index score was calculated using 1 point for history of myocardial infarction, peripheral vascular disease, cerebrovascular disease or diabetes without complications; 2 points for congestive heart failure, chronic obstructive pulmonary disease, mild liver disease or cancer; 3 points for dementia or connective tissue disease; 4 points for moderate to severe liver disease or HIV infection; and 6 points for metastatic cancer.
Figure 1.
Calibration curve for the LACE index, based on data representing patients in the derivation and internal validation groups. Note: bars = number of patients with the same LACE score; black line = expected risk of death or unplanned readmission within 30 days after discharge; grey line = observed risk (error bars = 95% confidence intervals).
The LACE index had moderate discrimination for early death or readmission. The C statistic (95% CI) in the derivation was 0.7114 (0.6736–0.7491). In the validation, it was 0.6935 (0.6548–07321), and in the entire cohort, it was 0.7025 (0.6755–0.7295).
The expected probability of death or readmission within 30 days of discharge for each point ranged from 2.0% for a LACE score of 0 to 43.7% for a LACE score of 19 (Table 4). The expected probability of early death or unplanned readmission was within the 95% CIs of the observed rates for all LACE scores in both the derivation and validation cohorts (Table 4) as well as the entire cohort (Figure 1). The Hosmer–Lemeshow statistic in the derivation was 18.7 (p = 0.42). In the validation, it was 14.1 (p = 0.59), and in the entire cohort, it was 21.2 (p = 0.27) (Table 4).
Table 4.
Expected and observed probability of death or unplanned readmission within 30 days after discharge, by LACE score
| LACE score | Expected probability, % | Observed probability, % (95% CI) | |
|---|---|---|---|
| Derivation group n = 2393 | Validation group n = 2419 | ||
| 0 | 2.0 | 0.0 (0.0–61.5) | 0.0 (0.0–46.1) |
| 1 | 2.5 | 1.4 (0.2–5.1) | 3.0 (0.8–7.6) |
| 2 | 3.0 | 2.6 (0.5–7.5) | 2.7 (0.5–7.8) |
| 3 | 3.5 | 5.6 (2.2–11.4) | 2.5 (0.5–7.2) |
| 4 | 4.3 | 3.9 (2.0–6.9) | 2.3 (0.9–4.8) |
| 5 | 5.1 | 4.4 (2.2–7.9) | 6.7 (3.9–10.8) |
| 6 | 6.1 | 4.7 (2.3–8.7) | 4.5 (2.0–8.5) |
| 7 | 7.3 | 7.6 (4.9–11.4) | 8.5 (5.8–12.0) |
| 8 | 8.7 | 6.3 (3.8–9.8) | 8.0 (4.9–12.2) |
| 9 | 10.3 | 11.7 (6.8–18.8) | 8.7 (5.0–14.2) |
| 10 | 12.2 | 14.5 (9.4–21.3) | 13.6 (8.7–20.2) |
| 11 | 14.4 | 18.6 (11.5–28.4) | 18.1 (10.9–28.3) |
| 12 | 17.0 | 20.8 (11.7–34.4) | 10.4 (4.5–20.5) |
| 13 | 19.8 | 17.3 (7.9–32.9) | 17.4 (7.5–34.3) |
| 14 | 23.0 | 28.6 (12.3–56.3) | 36.4 (15.7–71.7) |
| 15 | 26.6 | 8.3 (0.2–46.4) | 18.8 (3.9–54.8) |
| 16 | 30.4 | 50.0 (18.3–100) | 29.4 (9.6–68.6) |
| 17 | 34.6 | 33.3 (6.9–97.4) | 42.9 (8.8–100) |
| 18 | 39.1 | 100.0 (12.1–100) | – |
| 19 | 43.7 | 0.0 | – |
The LACE score was strongly associated with each outcome individually. A 1-point increase in the LACE score increased the odds of unplanned readmission by 18% (odds ratio 1.18, 95% CI 1.14–1.21). The LACE index in the entire cohort was moderately discriminative for 30-day unplanned readmission (C statistic 0.679, 95% CI 0.650–0.708) and well calibrated (Hosmer–Lemeshow statistic 11.5, p = 0.18). A one-point increase in the LACE score increased the odds of early death by 29% (odds ratio 1.29, 95% CI 1.20–1.38). The LACE index was very discriminative for early death (C statistic 0.793, 95% CI 0.733–0.854) and well calibrated (Hosmer–Lemeshow statistic 4.7, p = 0.79).
External validation
The external validation group contained 1 000 000 randomly selected patients (mean age 59.1, standard deviation [SD] 18.4 years; 48.4% female). Patients had a mean length of stay of 5.1 days (SD 7.7), a mean Charlson comorbidity index score of 0.5 (SD 1.2), and a mean number of emergency department visits of 0.4 (SD 7.9), with 67.6% of the index admissions emergent. Patients had a mean LACE score of 6.0 (SD 3.1) and 7.8% of patients died (1.1%) or were urgently readmitted (7.3%) within 30 days of discharge. Discrimination of the LACE index was the same in this patient group (C statistic 0.684, 95% CI 0.679–0.691). The observed rate of early death or urgent readmission slightly exceeded the expected rates at most LACE scores (Figure 2). However, the median absolute difference between expected and observed rates was small, at 1.6% (range 0.04%–6.6%).
Figure 2.
External validation of the LACE index, as represented by its accuracy for 1 000 000 randomly selected patients discharged from hospital in Ontario between 2004 and 2008. Note: bars = number of patients with the same LACE score; black line = expected risk of death or unplanned readmission within 30 days after discharge; grey line = observed risk (error bars = 95% confidence intervals).
Interpretation
We have derived and validated an easy-to-use index that is moderately discriminative and very accurate for predicting the risk of early death or unplanned readmission after discharge from hospital to the community. Further research is required to determine whether such quantification changes patient care or outcomes.
We found its simplicity very notable. Although we derived the LACE index in a large cohort of patients using almost 50 factors — each of which could reasonably influence the risk of post-discharge outcomes — we found that four simple factors explained much of the variation in risk of early death or unplanned readmission after discharge from hospital. The LACE index therefore joins other indexes in which seemingly complex outcomes are predicted with a few simple factors.23
The LACE index has several strengths to support its use.24 The outcome predicted by the index is important, clinically relevant and reliably measured. Determination of this outcome for each patient was independent of the LACE score. Each component of the LACE index is readily and reliably determined. The methods we used to derive the LACE index were both valid and transparent. The discrimination of the LACE index was better than that of the widely used Framing-ham score in many populations,25–27 which suggests that the LACE index will be useful when applied at the individual patient level. The calibration of the LACE index was excellent, which suggests that it will also be useful when applied by policy-makers. Finally, the LACE index is easier to use than previous models, because the latter require variables —such as community admission rates28 or area-level socio-economic measures29 — that are usually unavailable to clinicians.
Limitations
Three main limitations about the LACE index should be noted. First, the index cannot be used reliably in patient populations that were not involved in its derivation. Second, further work is required to identify additional factors that may increase the discrimination or accuracy of the index. Third, clinicians will find it difficult to commit to memory the point system and its expected risks. Therefore, use of the LACE index will usually require a computational aid. Until the LACE index is externally validated with primary data, we recommend that it be used for outcomes research and quality assurance rather than in decision-making for individual patients.
Conclusion
Notwithstanding its limitations, we believe that the LACE index can be used by researchers and administrators to predict the risk of early death or unplanned readmission of cognitively intact medical or surgical patients after discharge from hospital to the community. Further research is required to determine whether quantifying the risk of poor outcomes after discharge actually changes patient care or outcomes.
Supplementary Material
Footnotes
Funding: This study was funded by the Canadian Institutes of Health Research, the Physicians’ Services Incorporated Foundation and the Department of Medicine, University of Ottawa.
Previously published at www.cmaj.ca
Competing interests: None declared.
Contributors: All of the authors were involved in the conception and design of the study, the acquisition of data, the analysis or interpretation of data, the drafting of the manuscript and the critical revision of the manuscript for important intellectual content. All of them approved the final version submitted for publication. Dr. van Walraven had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
This article has been peer reviewed.
REFERENCES
- 1.Anderson GF, Steinberg EP. Hospital readmissions in the Medicare population. N Engl J Med. 1984;311:1349–53. doi: 10.1056/NEJM198411223112105. [DOI] [PubMed] [Google Scholar]
- 2.Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare Fee-for-Service Program. N Engl J Med. 2009;360:1418–28. doi: 10.1056/NEJMsa0803563. [DOI] [PubMed] [Google Scholar]
- 3.Naylor M, Brooten D, Jones R, et al. Comprehensive discharge planning for the hospitalized elderly. A randomized clinical trial. Ann Intern Med. 1994;120:999–1006. doi: 10.7326/0003-4819-120-12-199406150-00005. [DOI] [PubMed] [Google Scholar]
- 4.Coleman EA, Parry C, Chalmers S, et al. The Care Transitions Intervention: results of a randomized controlled trial. Arch Intern Med. 2006;166:1822–8. doi: 10.1001/archinte.166.17.1822. [DOI] [PubMed] [Google Scholar]
- 5.Rich MW, Beckham V, Wittenberg C, et al. A multidisciplinary intervention to prevent the readmission of elderly patients with congestive heart failure. N Engl J Med. 1995;333:1190–5. doi: 10.1056/NEJM199511023331806. [DOI] [PubMed] [Google Scholar]
- 6.Naylor MD, Brooten D, Campbell R, et al. Comprehensive discharge planning and home follow-up of hospitalized elders: a randomized clinical trial. JAMA. 1999;281:613–20. doi: 10.1001/jama.281.7.613. [DOI] [PubMed] [Google Scholar]
- 7.Jack BW, Chetty VK, Anthony D, et al. A reengineered hospital discharge program to decrease rehospitalization: a randomized trial. Ann Intern Med. 2009;150:178–87. doi: 10.7326/0003-4819-150-3-200902030-00007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Epstein AM. Revisiting readmissions — changing the incentives for shared accountability. N Engl J Med. 2009;360:1457–9. doi: 10.1056/NEJMe0901006. [DOI] [PubMed] [Google Scholar]
- 9.Report to the Congress: reforming the delivery system. Washington (DC): Medicare Payment Advisory Commission; 2008. A path to bundled payment around a rehospitalization; pp. 83–103. [Google Scholar]
- 10.Billings J, Dixon J, Mijanovich T, et al. Case finding for patients at risk of read-mission to hospital: development of algorithm to identify high risk patients. BMJ. 2006;333:327. doi: 10.1136/bmj.38870.657917.AE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Bottle A, Aylin P, Majeed A. Identifying patients at high risk of emergency hospital admissions: a logistic regression analysis. J R Soc Med. 2006;99:406–14. doi: 10.1258/jrsm.99.8.406. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Ashton CM, Wray NP. A conceptual framework for the study of early readmission as an indicator of quality of care. Soc Sci Med. 1996;43:1533–41. doi: 10.1016/s0277-9536(96)00049-4. [DOI] [PubMed] [Google Scholar]
- 13.Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–83. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
- 14.Schneeweiss S, Wang PS, Avorn J, et al. Improved comorbidity adjustment for predicting mortality in Medicare populations. Health Serv Res. 2003;38:1103–20. doi: 10.1111/1475-6773.00165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Royston P, Altman DG. Regression using fractional polynomials of continuous covariates: Parsimonious parametric modelling. Appl Stat. 1994;43:429–67. [Google Scholar]
- 16.Sauerbrei W, Royston P. Building multivariable prognostic and diagnostic models: transformation of the predictors by using fractional polynomials. J R Stat Soc [Ser A] 1999;162:71–94. [Google Scholar]
- 17.Sauerbrei W, Meier-Hirmer C, Benner A, et al. Multivariable regression model building by using fractional polynomials: Description of SAS, STATA and R programs. Comput Stat Data Anal. 2006;50:3464–85. [Google Scholar]
- 18.Sullivan LM, Massaro JM, D’Agostino RB., Sr Presentation of multivariate data for clinical use: the Framingham study risk score functions. Stat Med. 2004;23:1631–60. doi: 10.1002/sim.1742. [DOI] [PubMed] [Google Scholar]
- 19.Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43:1130–9. doi: 10.1097/01.mlr.0000182534.19832.83. [DOI] [PubMed] [Google Scholar]
- 20.Gonen M. Analyzing receiver operating characteristic curves with SAS. Cary (NC): SAS Institute Inc; 2007. Single continuous predictor; pp. 15–36. [Google Scholar]
- 21.Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. 3rd ed. Hoboken (NJ): John Wiley & Sons; 2003. Poisson regression; pp. 340–72. [Google Scholar]
- 22.Hosmer DW, Hosmer T, Le Cessie S, et al. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16:965–80. doi: 10.1002/(sici)1097-0258(19970515)16:9<965::aid-sim509>3.0.co;2-o. [DOI] [PubMed] [Google Scholar]
- 23.Lee TH, Marcantonio ER, Mangione CM, et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999;100:1043–9. doi: 10.1161/01.cir.100.10.1043. [DOI] [PubMed] [Google Scholar]
- 24.Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA. 1997;277:488–94. [PubMed] [Google Scholar]
- 25.Guzder RN, Gatling W, Mullee MA, et al. Prognostic value of the Framingham cardiovascular risk equation and the UKPDS risk engine for coronary heart disease in newly diagnosed type 2 diabetes: results from a United Kingdom study. Diabet Med. 2005;22:554–62. doi: 10.1111/j.1464-5491.2005.01494.x. [DOI] [PubMed] [Google Scholar]
- 26.Empana JP, Ducimetiere P, Arveiler D, et al. Are the Framingham and PROCAM coronary heart disease risk functions applicable to different European populations? The PRIME Study. [see comment] Eur Heart J. 2003;24:1903–11. doi: 10.1016/j.ehj.2003.09.002. [DOI] [PubMed] [Google Scholar]
- 27.Orford JL, Sesso HD, Stedman M, et al. A comparison of the Framingham and European Society of Cardiology coronary heart disease risk prediction models in the normative aging study. Am Heart J. 2002;144:95–100. doi: 10.1067/mhj.2002.123317. [DOI] [PubMed] [Google Scholar]
- 28.Billings J, Dixon J, Mijanovich T, et al. Case finding for patients at risk of readmission to hospital: development of algorithm to identify high risk patients. BMJ. 2006;333:327. doi: 10.1136/bmj.38870.657917.AE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Bottle A, Aylin P, Majeed A. Identifying patients at high risk of emergency hospital admissions: a logistic regression analysis. J R Soc Med. 2006;99:406–14. doi: 10.1258/jrsm.99.8.406. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.