Abstract
Background
Previous studies of hospital readmission have focused on specific conditions or populations and generated complex prediction models.
Objective
To identify predictors of early hospital readmission in a diverse patient population and derive and validate a simple model for identifying patients at high readmission risk.
Design
Prospective observational cohort study.
Patients
Participants encompassed 10,946 patients discharged home from general medicine services at six academic medical centers and were randomly divided into derivation (n = 7,287) and validation (n = 3,659) cohorts.
Measurements
We identified readmissions from administrative data and 30-day post-discharge telephone follow-up. Patient-level factors were grouped into four categories: sociodemographic factors, social support, health condition, and healthcare utilization. We performed logistic regression analysis to identify significant predictors of unplanned readmission within 30 days of discharge and developed a scoring system for estimating readmission risk.
Results
Approximately 17.5% of patients were readmitted in each cohort. Among patients in the derivation cohort, seven factors emerged as significant predictors of early readmission: insurance status, marital status, having a regular physician, Charlson comorbidity index, SF12 physical component score, ≥1 admission(s) within the last year, and current length of stay >2 days. A cumulative risk score of ≥25 points identified 5% of patients with a readmission risk of approximately 30% in each cohort. Model discrimination was fair with a c-statistic of 0.65 and 0.61 for the derivation and validation cohorts, respectively.
Conclusions
Select patient characteristics easily available shortly after admission can be used to identify a subset of patients at elevated risk of early readmission. This information may guide the efficient use of interventions to prevent readmission.
KEY WORDS: hospital, readmission, predictive, model
INTRODUCTION
Hospital readmission shortly after discharge is increasingly recognized as a marker of inpatient quality of care and a significant contributor to rising healthcare costs1,2. Nearly one fifth of Medicare beneficiaries discharged from acute care hospitals are readmitted within 30 days, incurring additional costs of several billion dollars annually3. Although it remains unclear whether such readmissions are entirely preventable,4–6 there is good evidence that targeted interventions initiated before and/or shortly after discharge can decrease the likelihood of readmission7–11. Identifying patients at risk of readmission can guide efficient resource utilization and permit valid comparisons of hospital quality across institutions.
Previous studies that have examined risk factors for early hospital readmission have focused primarily on a single disease or condition,12–15 a single hospital site,16–18 or a specific patient population19–22. Of four large multi-hospital studies that modeled readmission risk in a diverse patient population, one studied patients discharged from Veterans Affairs hospitals,23 two were conducted in England and employed extensive information technology resources unavailable in most other countries,24,25 and the fourth utilized Medicare data to derive a highly predictive but difficult to use model incorporating 20 variables26. These studies yielded complex prediction models that used patient information not currently easily available in most hospitals and did not adequately assess the impact of patients’ social supports on readmission risk.
To address these gaps, we used data from the Multicenter Hospitalist (MCH) Study to identify patient-level factors significantly associated with early hospital readmission among general medicine patients hospitalized in six large academic medical centers. In addition, we aimed to create and internally validate a simple score-based prediction model to identify patients with significantly elevated readmission risk. We limited our analysis to patient information that could be easily collected within the first 48 hours of admission.
METHODS
Context
The MCH Study was a prospective multi-center trial designed to assess the impact of hospitalist care on patients admitted to the general medicine services of six academic medical centers27–29. Patients were enrolled from July 1, 2001 through June 30, 2003 at the following six sites: University of Chicago, University of California San Francisco, University of Iowa, University of Wisconsin, University of New Mexico, and Brigham and Women’s Hospital in Boston. The study was approved by each site’s institutional review board.
Patients were eligible for inclusion if they were 18 years of age or older and were admitted by a hospitalist or other internist to a general medicine service. Patients admitted specifically under the care of their primary care physician were excluded.
Data Collection
Detailed sociodemographic and health information was collected during a 15–20 minute intake interview conducted by a research assistant, generally within 48 hours of admission. Additional data were obtained from each site’s administrative records and a telephone interview of patients or their proxies conducted 30 days after discharge. These data were matched with the National Death Index to ascertain 30-day mortality from the date of hospital discharge.
Administrative data were used to estimate length of stay and to ascertain age, sex, and insurance status. Intake interviews were used to administer the adult lifestyles and function interview mini-mental state exam (ALFI-MMSE),30,31 the Medical Outcomes Study Short Form 12 (SF12) questionnaire,32 and gather data on social supports, prior healthcare utilization, and health condition, including comorbidities for calculating a self-reported Charlson index33.
Patients
We retrospectively selected a subset of enrolled MCH Study patients for our analysis. First, we only included patients where they or their proxies could be interviewed in the hospital and therefore could provide timely data for our predictive models. We then excluded patients with a length of stay greater than 30 days to reduce bias from outlier effects. Next, we excluded patients not discharged to home, i.e., patients who died during hospitalization, were transferred to another healthcare facility, or left against medical advice. Lastly, we excluded patients who died within 30 days of discharge.
Outcome Variable
We defined hospital readmission as all-cause admission to an acute care hospital within 30 days of discharge from the index hospitalization. We identified readmissions in two ways: using administrative data from the study sites and from patient response to a specific question regarding hospital readmission included in the 30-day telephone follow-up. To minimize recall bias, administrative data were used to identify readmissions to each index hospital, while self-reported data were only used to identify readmissions to non-index hospitals.
Predictor Variables
We identified candidate patient factors likely to be associated with high readmission risk a priori from a survey of the relevant literature and grouped them into four natural categories as follows: (1) sociodemographic factors, including age, sex, self-reported race/ethnicity, self-reported total household income, education, and insurance status; (2) social support including, marital status, number of people living with patient, having someone to help at home, and having a regular physician; (3) health condition, including self-reported 0–9 Charlson comorbidity index, self-reported 0–100 health rating, 0–100 SF12 physical and mental component scores, 0–22 ALFI-MMSE score, and limitations in activities of daily living (ADLs) and/or instrumental activities of daily living (IADLs); and (4) healthcare utilization, including number of admissions in last one year, length of stay of the current hospital admission, and whether, given the choice, the patient would stay an extra day in the hospital even if their doctor told them they were well enough to go home.
Statistical Analysis
The patient was the unit of analysis. Because of our large sample size, we chose a split-sample design to derive and internally validate our prediction model. We randomly selected two thirds of patients from each site and combined them to create a derivation cohort and subsequently combined the remaining one third of patients from each site to create a validation cohort34.
To assess whether the candidate patient factors were significantly associated with hospital readmission, we fitted separate multivariable logistic regression models for each of the four categories of patient factors using data from the derivation cohort. We used P < 0.10 as the cutoff for assessing significance. Only factors noted to be significantly associated with readmission within their respective categories were included in the final regression model. Generalized estimating equations (GEE) were used to account for clustering by discharging physician, and hospital site was entered as a fixed effect in each of the models to minimize confounding35.
When constructing the final model, factors that became non-significant at P > 0.05 were removed if their presence did not change the beta-coefficient for any other factor by more than 20%. We derived a scoring system by multiplying each beta coefficient by ten and rounding to the nearest integer; the integer values from all applicable factors were then added together to estimate a total score for each patient. We subsequently obtained score-based predicted probabilities of readmission by entering each patient’s risk score into a single-predictor logistic regression model and used the output from this model to determine score cutoffs for identifying patients within selected readmission risk levels (0–9%, 10–19%, 20–29%, and 30% or higher).
We tested the performance of our model using data from the validation cohort. We assessed goodness of fit using the The Hosmer–Lemeshow chi-square test36 and model discrimination by measuring the C statistic, which is the area under the receiver operating characteristic (ROC) curve37. Because patients discharged to sub-acute or long-term care facilities are an important patient population but might have different predictors of readmission, we repeated our methodology in this population. We used SAS statistical software (Version 9.1; SAS Inc, North Carolina) to perform all analyses.
RESULTS
Patient selection is described in Fig. 1. Of 13,903 patients who failed to complete the intake interview, 28% refused to sign informed consent, 52% were discharged before they or their proxies could be interviewed, 17% had been admitted during the previous month and were not re-interviewed, and 3% died in the hospital. The 10,946 patients selected for our analysis were randomly assigned to a derivation cohort of 7,287 patients and a validation cohort of 3,659 patients. There were no statistically significant differences in patient characteristics between the two cohorts (Table 1).
Figure 1.
Patient selection.
Table 1.
Patient Characteristics
| Characteristica | Entire cohort n = 10,946 | Derivation cohortbn = 7,287 | Validation cohortbn = 3,659 |
|---|---|---|---|
| Readmitted, n (%) | 1,912 (17.5) | 1,274 (17.5) | 638 (17.4) |
| Male sex, n (%) | 4,774 (43.6) | 3,167 (43.5) | 1,607 (43.9) |
| Age group, n (%) | |||
| 18–39 years | 2,396 (21.9) | 1,590 (21.8) | 806 (22.0) |
| 40–60 years | 3,903 (35.7) | 2,597 (35.6) | 1,306 (35.7) |
| 61–75 years | 2,493 (22.8) | 1,681 (23.1) | 812 (22.2) |
| >75 years | 2,154 (19.7) | 1,419 (19.5) | 735 (20.1) |
| Race/ethnicity (n = 10,919), n (%) | |||
| White | 5,009 (45.9) | 3,318 (45.7) | 1,691 (46.3) |
| Black | 3,961 (36.3) | 2,635 (36.3) | 1,326 (36.3) |
| Asian | 558 (5.1) | 373 (5.1) | 185 (5.1) |
| Other | 638 (5.8) | 432 (5.9) | 206 (5.6) |
| Hispanic | 753 (6.9) | 511 (7.0) | 242 (6.6) |
| Household income in dollars, n (%) | |||
| ≤ 15,000 | 2,683 (24.5) | 1,798 (24.7) | 885 (24.2) |
| 15,001–35,000 | 1,353 (12.4) | 906 (12.4) | 447 (12.2) |
| 35,001–50,000 | 697 (6.4) | 458 (6.3) | 239 (6.5) |
| > 50,000 | 1,405 (12.8) | 947 (13.0) | 458 (12.5) |
| Do not know or missing | 4,808 (43.9) | 3,178 (43.6) | 1,630 (44.6) |
| Education (n = 10,316), n (%) | |||
| < High school | 2,250 (21.8) | 1,518 (22.1) | 732 (21.2) |
| High school graduate | 3,085 (29.9) | 2,042 (29.7) | 1043 (30.2) |
| Some college | 2,651 (25.7) | 1,772 (25.8) | 879 (25.5) |
| ≥ College graduate | 2,330 (22.6) | 1,534 (22.3) | 796 (23.1) |
| Primary insurance (n = 10,465), n (%) | |||
| Medicare | 4,687 (44.8) | 3,105 (44.5) | 1,582 (45.3) |
| Medicaid | 1,973 (18.9) | 1,314 (18.8) | 659 (18.9) |
| Self-pay | 2,536 (24.2) | 1,708 (24.5) | 828 (23.7) |
| Private | 1,269 (12.1) | 846 (12.1) | 423 (12.1) |
| Marital status (n = 10,553), n (%) | |||
| Currently married | 4,019 (38.1) | 2,700 (38.4) | 1,319 (37.4) |
| Not currently married | 6,534 (61.9) | 4,326 (61.6) | 2,208 (62.6) |
| No. of people live with (n = 10,165), n (%) | |||
| Alone | 2,210 (21.7) | 1,461 (21.7) | 749 (21.9) |
| ≥ 1 | 7,955 (78.3) | 5,286 (78.4) | 2,669 (78.1) |
| Someone to help (n = 10,596), n (%) | |||
| Yes | 9561 (90.2) | 6,359 (90.1) | 3,202 (90.5) |
| No | 1,035 (9.8) | 697 (9.9) | 338 (9.6) |
| Regular physician (n = 10,793), n (%) | |||
| Yes | 8,659 (80.2) | 5,762 (80.2) | 2,897 (80.2) |
| No | 2,134 (19.8) | 1,420 (19.8) | 714 (19.8) |
| Charlson index (n = 10,630), median (IQR) | 1 (0–2) | 1 (0–2) | 1 (0–2) |
| Self-rated health (n = 10,146), mean (SD) | 55.2 (24.9) | 55.2 (25.0) | 55.1 (24.9) |
| Physical SF12 (n = 9,920), mean (SD) | 38.2 (12.7) | 38.3 (12.7) | 37.8 (12.7) |
| Mental SF12 (n = 9,920), mean (SD) | 48.2 (11.6) | 48.1 (11.6) | 48.2 (11.5) |
| Mini-mental state (n = 8,796), mean (SD) | 20.3 (2.4) | 20.3 (2.4) | 20.3 (2.5) |
| Functional limitations (n = 10,166), n (%) | |||
| No help required | 5,328 (52.4) | 3,558 (52.6) | 1,770 (52.0) |
| Little help with IADLs only | 1,428 (14.1) | 972 (14.4) | 456 (13.4) |
| Lots of help with IADLs only | 903 (8.9) | 596 (8.8) | 307 (9.0) |
| Little help with ADLs | 1,259 (12.4) | 830 (12.3) | 429 (12.6) |
| Lots of help with ADLs | 1,248 (12.3) | 805 (11.9) | 443 (13.0) |
| Admissions in last year (n = 10,772), n (%) | |||
| None | 5,397 (50.1) | 3,602 (50.3) | 1,795 (49.8) |
| 1 | 2,493 (23.1) | 1628 (22.7) | 865 (24.0) |
| 2 | 1,178 (10.9) | 782 (10.9) | 396 (11.0) |
| 3 | 682 (6.3) | 454 (6.3) | 228 (6.3) |
| 4 | 360 (3.3) | 240 (3.4) | 120 (3.3) |
| ≥ 5 | 662 (6.2) | 461 (6.4) | 201 (5.6) |
| Current length of stay, n (%) | |||
| 1–2 days | 4,532 (41.4) | 3,010 (41.3) | 1,522 (41.6) |
| > 2 days | 6,414 (58.6) | 4,277 (58.7) | 2,137 (58.4) |
| Stay extra day (n = 10,519), n (%) | |||
| Yes | 3,879 (36.9) | 3,920 (56.0) | 1,934 (54.9) |
| No | 5,854 (55.7) | 2,571 (36.7) | 1,308 (37.2) |
| Don’t know | 786 (7.5) | 508 (7.3) | 278 (7.9) |
| Site of enrollment, n (%) | |||
| (A) | 3,704 (33.8) | 2,463 (33.8) | 1,241 (33.9) |
| (B) | 1,267 (11.6) | 847 (11.6) | 420 (11.5) |
| (C) | 2,994 (27.4) | 1,990 (27.3) | 1,004 (27.4) |
| (D) | 1,663 (15.2) | 1,110 (15.2) | 553 (15.1) |
| (E) | 1,318 (12.0) | 877 (12.0) | 441 (12.1) |
a Because of rounding, percentages may not equal 100; because of missing data, number of patients in some categories may be less than the entire cohort (shown in parentheses)
b All P values for differences between derivation cohort and validation cohort were >0.05 using the chi-square test for categorical variables and t-test for continuous variables
Approximately 20% of patients in each cohort were older than 75 years, over 60% were not currently married, approximately one-quarter needed at least some help with their ADLs, and half had been hospitalized at least once in the preceding year. The mean rating for self-rated health (one month prior to admission) was 55 (standard deviation 25), and the median Charlson comorbidity index was 1 (inter-quartile range 0–2).
Of the 7,287 patients in the derivation cohort, 1,274 (17.5%) were readmitted within 30 days, of which 79% could be confirmed with administrative data from the index hospitals. Readmission rates varied from 16.1% to 17.9% among the different sites. Table 2 compares readmitted and non-readmitted patients and shows the results of each of the four sub-models used to derive the final model. The significant predictors included sociodemographic factors (age, income, insurance status), social support factors (marital status, having a regular physician), markers of health (Charlson comorbidity index, SF12 physical component score), and healthcare utilization factors (number of admissions in last one year, current length of stay greater than two days).
Table 2.
Association of Patient Characteristics with 30-Day Hospital Readmission in the Derivation Cohort
| Category | Characteristica | Readmitted n = 1,274 | Not readmitted n = 6,013 | Odds ratio (95% CI)b | P valueb |
|---|---|---|---|---|---|
| Socio-demographic factors | Male sex (%) | 41.8 | 43.8 | 0.94 (0.84–1.06) | 0.30 |
| Age group (%) | |||||
| 18–39 years | 20.1 | 22.2 | 1.14 (0.89–1.46) | 0.30 | |
| 40–60 years | 34.9 | 35.8 | 1.20 (0.96–1.50) | 0.11 | |
| 61–75 years | 24.9 | 22.7 | 1.18 (0.98–1.41) | 0.08 | |
| > 75 years | 20.2 | 19.3 | Reference | ||
| Race/ethnicity (%) | |||||
| White | 44.2 | 46.0 | Reference | ||
| Black | 38.8 | 35.7 | 1.11 (0.90–1.38) | 0.34 | |
| Asian | 4.4 | 5.3 | 0.88 (0.65–1.19) | 0.41 | |
| Other | 5.7 | 6.0 | 0.94 (0.70–1.27) | 0.69 | |
| Hispanic | 7.0 | 7.0 | 1.04 (0.78–1.39) | 0.81 | |
| Income in dollars (%) | |||||
| ≤ 15,000 | 23.6 | 24.9 | 0.90 (0.70–1.17) | 0.44 | |
| 15,001–35,000 | 13.9 | 12.1 | 1.25 (0.96–1.64) | 0.09 | |
| 35,001–50,000 | 6.1 | 6.3 | 1.09 (0.81–1.48) | 0.56 | |
| > 50,000 | 11.9 | 13.2 | Reference | ||
| Do not know or missing | 44.6 | 43.4 | 1.02 (0.80–1.29) | 0.90 | |
| Education (%) | |||||
| < High school | 23.6 | 21.8 | 1.02 (0.84–1.25) | 0.84 | |
| High school graduate | 28.5 | 30.0 | 0.92 (0.75–1.12) | 0.40 | |
| Some college | 25.7 | 25.8 | 0.97 (0.80–1.18) | 0.77 | |
| ≥ College graduate | 22.2 | 22.4 | Reference | ||
| Insurance (%) | |||||
| Medicare | 50.7 | 43.2 | 2.22 (1.73–2.84) | < 0.001 | |
| Medicaid | 20.2 | 18.6 | 1.94 (1.50–2.51) | < 0.001 | |
| Self-pay | 21.2 | 25.2 | 1.53 (1.16–2.02) | 0.003 | |
| Private | 7.9 | 13.0 | Reference | ||
| Social support | Marital status (%) | ||||
| Currently married | 41.8 | 37.7 | 1.19 (1.04–1.36) | 0.01 | |
| Not currently married | 58.2 | 62.3 | Reference | ||
| No. of people live with (%) | |||||
| Alone | 21.0 | 21.8 | 1.04 (0.88–1.23) | 0.65 | |
| ≥ 1 | 79.0 | 78.2 | Reference | ||
| Someone to help (%) | |||||
| Yes | 91.4 | 89.9 | 1.15 (0.92–1.44) | 0.22 | |
| No | 8.6 | 10.2 | Reference | ||
| Regular physician (%) | |||||
| Yes | 84.0 | 79.4 | 1.44 (1.19–1.75) | < 0.001 | |
| No | 16.0 | 20.6 | Reference | ||
| Health condition | Charlson index, median (IQR) | 1 (0–2) | 1 (0–2) | 1.13 (1.08–1.19) (per 1 unit change) | < 0.001 |
| Self-rated health, mean (SD) | 52.0 (24.9) | 55.8 (24.9) | 0.98 (0.95–1.02) (per 10 unit change) | 0.31 | |
| Physical SF12, mean (SD) | 35.9 (12.4) | 38.8 (12.7) | 0.90 (0.83–0.98) (per 10 unit change) | 0.01 | |
| Mental SF12, mean (SD) | 47.2 (12.0) | 48.3 (11.5) | 0.97 (0.90–1.04) (per 10 unit change) | 0.35 | |
| Mini mental state, mean (SD) | 20.2 (2.5) | 20.3 (2.3) | 0.85 (0.64–1.12) (per 10 unit change) | 0.25 | |
| Functional limitations (%) | |||||
| No help required | 44.2 | 54.4 | Reference | ||
| Little help with IADLs only | 14.7 | 14.3 | 0.99 (0.77–1.26) | 0.91 | |
| Lots of help with IADLs only | 10.4 | 8.5 | 1.16 (0.85–1.59) | 0.34 | |
| Little help with ADLs | 14.7 | 11.8 | 1.22 (0.91–1.64) | 0.18 | |
| Lots of help with ADLs | 16.0 | 11.0 | 1.20 (0.88–1.64) | 0.24 | |
| Healthcare utilization | Admissions in last year (%) | ||||
| None | 38.1 | 52.8 | Reference | ||
| 1 | 25.3 | 22.2 | 1.61 (1.36–1.89) | < 0.001 | |
| 2 | 12.9 | 10.5 | 1.71 (1.39–2.10) | < 0.001 | |
| 3 | 6.6 | 6.3 | 1.47 (1.13–1.92) | 0.005 | |
| 4 | 5.4 | 2.9 | 2.57 (1.89–3.51) | < 0.001 | |
| ≥ 5 | 11.8 | 5.3 | 3.02 (2.49–3.66) | < 0.001 | |
| Current length of stay (%) | |||||
| 1–2 days | 33.8 | 42.9 | Reference | ||
| > 2 days | 66.3 | 57.1 | 1.42 (1.25–1.60) | < 0.001 | |
| Stay extra day (%) | |||||
| Yes | 53.7 | 56.5 | 1.03 (0.89–1.19) | 0.69 | |
| No | 39.2 | 36.2 | Reference | ||
| Don’t know | 7.2 | 7.3 | 0.96 (0.75–1.22) | 0.71 |
a Because of rounding, percentages may not equal 100.
b Odds ratios and P values were derived from separate multivariable logistic regression models for each category; generalized estimating equations were used to account for clustering by discharging physician and hospital site was entered as a fixed effect in each model.
In the final model, two previously significant predictors—age and income—lost statistical significance (Table 3). Removing either of these predictors did not change the beta coefficients of other predictors by more than 20%, and so both were removed from the final model. As a consequence of removing these two, SF12 physical component score became non-significant; however, removing this predictor caused a substantial change in the beta coefficients of remaining predictors and it was retained in the final model. Study site was retained as an obligatory confounder. The odds ratio for the site with the highest adjusted readmission rate was 1.40 (95% CI 1.09–1.79) compared with the site with the lowest rate; no other differences among sites were statistically significant.
Table 3.
Final Logistic Regression Model of Predictors of 30-Day Hospital Readmissiona
| Variable | Beta coefficient | Odds ratio (95% CI) | P value | Pointsb |
|---|---|---|---|---|
| Insurance | ||||
| Medicare | 0.549 | 1.73 (1.37–2.19) | < 0.001 | 5 |
| Medicaid | 0.419 | 1.52 (1.14–2.03) | 0.004 | 4 |
| Self-pay | 0.435 | 1.55 (1.15–2.07) | 0.004 | 4 |
| Private | Reference | Reference | 0 | |
| Currently married | 0.216 | 1.24 (1.09–1.41) | 0.001 | 2 |
| Have a regular physician | 0.288 | 1.33 (1.09–1.64) | 0.006 | 3 |
| Charlson index | 0.090 | 1.09 (1.05–1.14) | < 0.001 | 1/unit |
| Physical SF12 | −0.007 | 0.99 (0.99–1.00) | 0.01 | −1/10 units |
| Admissions in last one year | ||||
| None | Reference | Reference | 0 | |
| 1 | 0.452 | 1.57 (1.31–1.88) | < 0.001 | 4 |
| 2 | 0.489 | 1.63 (1.31–2.03) | < 0.001 | 4 |
| 3 | 0.157 | 1.17 (0.87–1.56) | 0.29 | 4 |
| 4 | 0.858 | 2.36 (1.65–3.36) | < 0.001 | 9 |
| ≥ 5 | 1.077 | 2.94 (2.36–3.66) | < 0.001 | 11 |
| Current length of stay >2 days | 0.301 | 1.35 (1.18–1.54) | < 0.001 | 3 |
a Generalized estimating equations were used to account for clustering by discharging physician and hospital site was retained as a fixed effect in the model
b Calculated by multiplying beta coefficient by 10 and rounding to the nearest integer (with exception for “Admissions in last one year”)
Points were assigned to each predictor as described in the Methods section, except four points were assigned for ‘3 admissions in last one year’ to allow for a monotonic function. Using a regression model based only on the scoring system, we were able to assign score cutoffs based on predicted readmission rates of 0–9%, 10–19%, 20–29%, and 30% or higher (Table 4). The 5.1% of patients with a score of 25 or higher had 30-day readmission rates of 32.6% and 28.9% in the derivation and validation sets, respectively, compared with a 30-day readmission rate of 16.4% in patients with a score below 25 (same rate in both cohorts).
Table 4.
Comparison of Score Predicted and Observed Readmission Rates
| Score range | ||||
|---|---|---|---|---|
| 0 to 6 | 7 to 17 | 18 to 24 | ≥ 25 | |
| % Patients in score rangea | 3.7 | 69.0 | 22.3 | 5.1 |
| Predicted % readmission rate | 0–9% | 10–19% | 20–29% | ≥ 30% |
| Observed % readmitted in derivation cohort | 9.8 | 14.6 | 23.0 | 32.6 |
| Observed % readmitted in validation cohort | 5.9 | 15.3 | 21.2 | 28.9 |
aBecause of rounding, percentages may not equal 100.
The Hosmer–Lemeshow goodness of fit test yielded P-values of 0.44 and 0.23 for the derivation and validation cohorts, respectively, indicating good model fit. Discrimination of the model was only fair: the area under the ROC curve (AUC) was 0.65 in the derivation cohort and 0.61 in the validation cohort (Fig. 2).
Figure 2.
Comparison of the receiver operating characteristic (ROC) curves for the derivation and validation cohorts. A. ROC curve for derivation cohort. B. ROC curve for validation cohort.
In a similar analysis in patients discharged to sub-acute or long-term care facilities, the only significant predictors of readmission were the number of hospital admissions in the preceding year and patient age (results not shown). The effects of Charlson comorbidity index and SF12 physical component score on readmission were of a similar magnitude as with patients discharged home but were not significant predictors due to wider confidence intervals. Hospital length of stay, marital status, presence of a PCP, and insurance status were much less predictive of readmission.
DISCUSSION
Using data from the MCH Study, we were able to identify key patient-level predictors of early hospital readmission and derive and internally validate a parsimonious and easy-to-use model for assessing readmission risk in general medicine patients hospitalized for a variety of medical conditions and discharged home. Using seven easily available predictors, our model was able to identify 5% of patients with an approximately 30% risk of readmission within 30 days of discharge. Although the discriminative ability of our model is only fair, it still provides a useful and easily applicable tool for identifying high-risk patients who may require more intensive use of hospital resources designed to reduce readmission rates.
Several patient-level factors identified as significant predictors were known from the published literature, such as the number of hospital admissions in the preceding year and the Charlson comorbidity index16–26. It was somewhat surprising that marital status and having a regular physician were both positively associated with readmission risk. It is possible that the presence of social supports, such as a spouse, allows some frail patients to be discharged home who would otherwise be transferred to a subacute care facility (of note, when our model was applied to patients discharged to facilities, being married was a negligible predictor of readmission). Similarly, having a regular physician may be a marker of illness severity not captured by other predictors in our model. Another possibility is that having a spouse or regular physician may lead to earlier detection of clinical deterioration and/or a lower threshold for readmission, although the lack of a significant association between having help at home or living with someone and early readmission appears to argue against this. Nevertheless, these explanations do not diminish the usefulness of these factors as predictors in our model. Future studies should validate these findings and compare model discrimination with and without these social factors.
Age is one predictor that has been noted to have a significant association with readmission risk in several studies but was non-significant in our final model. However, Medicare as primary insurance was a significant predictor in our final model and this variable incorporates age, which may partly explain these findings. Also, other predictive factors more commonly found in the elderly may already have been captured in our model, including Charlson comorbidity index and hospitalizations during the preceding one year.
In a supplementary analysis of patients discharged to sub-acute and long-term care facilities, we found that sociodemographic and social support variables were much less predictive of readmission. This may be because these factors mediate readmission through access to care and performance of self-care activities, and these factors are much less important when access to care is essentially continuous.
The performance characteristics of our model are only fair. However, they are comparable to the discriminative ability of other commonly cited readmission risk prediction models. Our AUC of 0.61 in the validation cohort is identical to the AUC of the Pra (probability of repeated admission) model21,38. Similar to our model, the Pra model was able to identify a small group (7.2%) of patients at high risk of readmission (41.8% with two or more admissions over 4 years). The one well-known model with a high AUC (0.83) included 20 variables with eight interaction terms, raising the possibility that it was overfit for the population used to derive it and limiting its usefulness as a practical clinical tool26. Two other high performing but similarly complicated models with AUC ranging from 0.68 to 0.75 were derived from the United Kingdom’s National Health Service database and are not usable in the U.S. due to their reliance on data from national electronic medical records24,25.
Why is it that so few statistical models derived to date are capable of reliably predicting readmission risk in a diverse population of medical inpatients? There are several possibilities. First, several important and previously unknown predictors may be missing from existing models. For example, we now know that adverse drug events (ADEs) are an important patient safety problem following hospital discharge, and existing models—including the one derived for this study—do not include many of the recently identified predictors of post-discharge ADEs (such as the number and classes of preadmission medications and patients’ knowledge of their medications)39,40. Second, generic markers of illness severity may be less predictive when evaluating populations with diverse medical conditions in contrast to disease-specific markers such as those for congestive heart failure12–14. Third, it is plausible that readmission risk has a weaker correlation with patients’ clinical characteristics and social circumstances than it does with the processes of care during hospitalization and discharge and with post-discharge care41. That adjusted readmission rate varied by site is one piece of evidence in favor of this hypothesis. Thus, rather than identifying a single group of patients at high risk of readmission and focusing interventions on them, it may be more efficient to ensure that all patients receive a standardized set of discharge processes11. Alternatively, it may be worth identifying different types of high-risk patients and customizing interventions accordingly (e.g., a focused pharmacist intervention for patients at high risk for ADEs; close follow-up for patients with certain high-risk medical conditions).
Our study has several limitations. Although it was conducted at six academic medical centers in different states and included detailed information on a sizeable and diverse patient population, caution should be exercised in generalizing its findings to small, rural, and/or community hospitals. A sizeable proportion of screened patients could not be included in our study, further limiting generalizability; analyzing readmissions in this population, especially in patients who were discharged before they could be interviewed, may yield additional insights into the reasons behind early hospital readmission in patients with short lengths of stay. Furthermore, we excluded patients who died within 30 days of discharge because predictors of death may be somewhat different than predictors of readmission. Since we did not adjudicate whether each readmission was elective versus unplanned, we could not exclude purely elective readmissions; however, based on our collective experience we would expect the rate of elective readmissions on general medicine services to be low. Lastly, we were unable to confirm readmissions to non-study hospitals, and patients are known to underreport hospital readmissions 42. However, the short time-frame (only 30 days) for measuring readmissions and use of administrative data to confirm readmissions to study hospitals minimizes the potential impact of recall bias.
In summary, a prediction model derived and internally validated in a large multi-center cohort of general medicine inpatients successfully identified a small proportion of patients at elevated risk of hospital readmission within 30 days of home discharge. While interventions could be designed and tested on this population, more work is needed to identify additional factors that impact post-discharge health outcomes, optimize the discharge process for all patients, and create interventions tailored to patients’ needs in order to prevent potentially avoidable readmissions.
Acknowledgments
Funding/support The Multicenter Hospitalist Study was supported by grant R01-HS10597 from the Agency for Healthcare Research and Quality. Dr. Hasan was supported by grant T32-HP11001 from the Health Resources and Services Administration of the Department of Health and Human Services to support the Harvard Medical School Fellowship in General Medicine and Primary Care.
The Multicenter Hospitalist Study is registered at http://clinicaltrials.gov (NCT00204048).
Conflict of Interest None disclosed.
References
- 1.McDonald KM, Sundaram V, Bravata DM, et al. Closing the quality gap: a critical analysis of quality improvement strategies. Volume 7: Care Coordination. AHRQ Publication No. 04(07)-0051-7. Rockville, MD: Agency for Healthcare Research and Quality; 2007. Available at www.ahrq.gov/downloads/pub/evidence/pdf/caregap/caregap.pdf. Accessed November 9, 2009. [PubMed]
- 2.Balla U, Malnick S, Schattner A. Early readmissions to the department of medicine as a screening tool for monitoring quality of care problems. Medicine. 2008;87:294–300. doi: 10.1097/MD.0b013e3181886f93. [DOI] [PubMed] [Google Scholar]
- 3.Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360:1418–1428. doi: 10.1056/NEJMsa0803563. [DOI] [PubMed] [Google Scholar]
- 4.Clarke A. Are readmissions avoidable? BMJ. 1990;301:1136–1138. doi: 10.1136/bmj.301.6761.1136. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Frankl SE, Breeling JL, Goldman L. Preventability of emergent hospital readmission. Am J Med. 1991;90:667–674. [PubMed] [Google Scholar]
- 6.Gautam P, Macduff C, Brown I, Squair J. Unplanned readmissions of elderly patients. Health Bull (Edinb) 1996;54:449–457. [PubMed] [Google Scholar]
- 7.Rich MW, Beckham V, Wittenberg C, Leven CL, Freedland KE, Carney RM. A multidisciplinary intervention to prevent the readmission of elderly patients with congestive heart failure. N Engl J Med. 1995;333:1190–1195. doi: 10.1056/NEJM199511023331806. [DOI] [PubMed] [Google Scholar]
- 8.Naylor MD, Brooten D, Campbell R, et al. Comprehensive discharge planning and home follow-up of hospitalized elders: a randomized clinical trial. JAMA. 1999;281:613–620. doi: 10.1001/jama.281.7.613. [DOI] [PubMed] [Google Scholar]
- 9.Naylor MD, McCauley KM. The effects of a discharge planning and home follow-up intervention on elders hospitalized with common medical and surgical cardiac conditions. J Cardiovasc Nurs. 1999;14:44–54. doi: 10.1097/00005082-199910000-00006. [DOI] [PubMed] [Google Scholar]
- 10.Naylor MD, Brooten DA, Campbell RL, Maislin G, McCauley KM, Schwartz JS. Transitional care of older adults hospitalized with heart failure: a randomized, controlled trial. J Am Geriatr Soc. 2004;52:675–684. doi: 10.1111/j.1532-5415.2004.52202.x. [DOI] [PubMed] [Google Scholar]
- 11.Jack BW, Chetty VK, Anthony D, et al. A reengineered hospital discharge program to decrease rehospitalization: a randomized trial. Ann Intern Med. 2009;150:178–187. doi: 10.7326/0003-4819-150-3-200902030-00007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Krumholz HM, Parent EM, Tu N, et al. Readmission after hospitalization for congestive heart failure among Medicare beneficiaries. Arch Intern Med. 1997;157:99–104. doi: 10.1001/archinte.157.1.99. [DOI] [PubMed] [Google Scholar]
- 13.Kossovsky MP, Sarasin FP, Perneger TV, Chopard P, Sigaud P, Gaspoz J. Unplanned readmissions of patients with congestive heart failure: do they reflect in-hospital quality of care or patient characteristics? Am J Med. 2000;109:386–390. doi: 10.1016/S0002-9343(00)00489-7. [DOI] [PubMed] [Google Scholar]
- 14.Chin MH, Goldman L. Correlates of early hospital readmission or death in patients with congestive heart failure. Am J Cardiol. 1997;79:1640–1644. doi: 10.1016/S0002-9149(97)00214-2. [DOI] [PubMed] [Google Scholar]
- 15.Grant RW, Charlebois ED, Wachter RM. Risk factors for early hospital readmission in patients with AIDS and pneumonia. J Gen Intern Med. 1999;14:531–536. doi: 10.1046/j.1525-1497.1999.08157.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Phillips RS, Safran C, Cleary PD, Delbanco TL. Predicting emergency readmissions for patients discharged from the medical service of a teaching hospital. J Gen Intern Med. 1987;2:400–405. doi: 10.1007/BF02596366. [DOI] [PubMed] [Google Scholar]
- 17.Corrigan JM, Martin JB. Identification of factors associated with hospital readmission and development of a predictive model. Health Serv Res. 1992;27:81–101. [PMC free article] [PubMed] [Google Scholar]
- 18.Smith DM, Katz BP, Huster GA, Fitzgerald JF, Martin DK, Freedman JA. Risk factors for nonelective hospital readmissions. J Gen Intern Med. 1996;11:762–764. doi: 10.1007/BF02598996. [DOI] [PubMed] [Google Scholar]
- 19.Holloway JJ, Medendorp SV, Bromberg J. Risk factors for early readmissions among veterans. Health Serv Res. 1990;25:213–237. [PMC free article] [PubMed] [Google Scholar]
- 20.Reed RL, Pearlman RA, Buchner DM. Risk factors for early unplanned hospital readmission in the elderly. J Gen Intern Med. 1991;6:223–228. doi: 10.1007/BF02598964. [DOI] [PubMed] [Google Scholar]
- 21.Boult C, Dowd B, McCaffrey D, Boult L, Hernandez R, Krulewitch H. Screening elders for risk of hospital admission. J Am Geriatr Soc. 1993;41:811–817. doi: 10.1111/j.1532-5415.1993.tb06175.x. [DOI] [PubMed] [Google Scholar]
- 22.Marcantonio ER, McKean S, Goldfinger M, Kleefield S, Yurkofsky M, Brennan TA. Factors associated with unplanned hospital readmission among patients 65 years of age and older in a Medicare managed care plan. Am J Med. 1999;107:13–17. doi: 10.1016/S0002-9343(99)00159-X. [DOI] [PubMed] [Google Scholar]
- 23.Smith DM, Giobbie-Hurder A, Weinberger M, et al. Predicting non-elective hospital readmissions: a multi-site study. Department of Veterans Affairs Cooperative Study Group on Primary Care and Readmissions. J Clin Epidemiol. 2000;53:1113–1118. doi: 10.1016/S0895-4356(00)00236-5. [DOI] [PubMed] [Google Scholar]
- 24.Billings J, Dixon J, Mijanovich T, Wennberg D. Case finding for patients at risk of readmission to hospital: development of algorithm to identify high risk patients. BMJ. 2006;333(7563):327. doi: 10.1136/bmj.38870.657917.AE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bottle A, Aylin P, Majeed A. Identifying patients at high risk of emergency hospital admissions: a logistic regression analysis. J R Soc Med. 2006;99:406–414. doi: 10.1258/jrsm.99.8.406. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Coleman EA, Min SJ, Chomiak A, Kramer AM. Posthospital care transitions: patterns, complications, and risk identification. Health Serv Res. 2004;39:1449–1465. doi: 10.1111/j.1475-6773.2004.00298.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Meltzer D, Arora V, Zhang J, et al. Effects of inpatient experience on outcomes and costs in a multicenter trial of academic hospitalists. J Gen Intern Med. 2005;20(s1):141–142. [Google Scholar]
- 28.Meltzer D, Wetterneck T, Kaboli P, et al. Effects of hospitalists on outcomes and costs in a multicenter trial of academic hospitalists. Paper presented to the annual meeting of the Society of Hospital Medicine, Chicago, April 2005.
- 29.Schneider JA, Zhang Q, Auerbach A, et al. Do hospitalists or physicians with greater inpatient HIV experience improve HIV care in the era of highly active antiretroviral therapy? Results from a multicenter trial of academic hospitalists. Clin Infect Dis. 2008;46:1085–1092. doi: 10.1086/529200. [DOI] [PubMed] [Google Scholar]
- 30.Roccaforte WH, Burke WJ, Bayer BL, Wengel SP. Validation of a telephone version of the mini-mental state examination. J Am Geriatr Soc. 1992;40:697–702. doi: 10.1111/j.1532-5415.1992.tb01962.x. [DOI] [PubMed] [Google Scholar]
- 31.Fischbach R. Early identification of demented persons in the community. In: Becker R, Giacobini E, editors. Alzheimer’s disease: current research in early diagnosis. New York, NY: Taylor and Francis Inc; 1990. pp. 49–74. [Google Scholar]
- 32.Ware J, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34:220–233. doi: 10.1097/00005650-199603000-00003. [DOI] [PubMed] [Google Scholar]
- 33.Katz JN, Chang LC, Sangha O, Fossel AH, Bates DW. Can comorbidity be measured by questionnaire rather than medical record review? Med Care. 1996;34:73–84. doi: 10.1097/00005650-199601000-00006. [DOI] [PubMed] [Google Scholar]
- 34.Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: applications and methodological standards. N Engl J Med. 1985;313:793–799. doi: 10.1056/NEJM198509263131306. [DOI] [PubMed] [Google Scholar]
- 35.Hanley JA, Negassa A, Edwardes MD, Forrester JE. Statistical analysis of correlated data using generalized estimating equations: an orientation. Am J Epidemiol. 2003;157:364–375. doi: 10.1093/aje/kwf215. [DOI] [PubMed] [Google Scholar]
- 36.Hosmer DW, Hosmer T, Cessie S, Lemeshow S. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16:965–980. doi: 10.1002/(SICI)1097-0258(19970515)16:9<965::AID-SIM509>3.0.CO;2-O. [DOI] [PubMed] [Google Scholar]
- 37.Centor RM, Schwartz JS. An evaluation of methods for estimating the area under the receiver operating characteristic (ROC) curve. Med Decis Making. 1985;5:149–156. doi: 10.1177/0272989X8500500204. [DOI] [PubMed] [Google Scholar]
- 38.Novotny NL, Anderson MA. Prediction of early readmission in medical inpatients using the Probability of Repeated Admission instrument. Nurs Res. 2008;57:406–415. doi: 10.1097/NNR.0b013e31818c3e06. [DOI] [PubMed] [Google Scholar]
- 39.Pippins JR, Gandhi TK, Hamann C, et al. Classifying and predicting errors of inpatient medication reconciliation. J Gen Intern Med. Sep 2008;23:1414–22. [DOI] [PMC free article] [PubMed]
- 40.Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. Adverse drug events occurring following hospital discharge. J Gen Intern Med. 2005;20:317–323. doi: 10.1111/j.1525-1497.2005.30390.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Polanczyk CA, Newton C, Dec GW, Salvo TG. Quality of care and hospital readmission in congestive heart failure: an explicit review process. J Card Fail. 2001;7:289–298. doi: 10.1054/jcaf.2001.28931. [DOI] [PubMed] [Google Scholar]
- 42.Norrish A, North D, Kirkman P, Jackson R. Validity of self-reported hospital admission in a prospective study. Am J Epidemiol. 1994;140:938–942. doi: 10.1093/oxfordjournals.aje.a117182. [DOI] [PubMed] [Google Scholar]