The HOSPITAL score predicts potentially preventable 30-day readmissions in conditions targeted by the Hospital Readmissions Reduction Program

Abstract

Background/Objectives

New tools to accurately identify potentially preventable 30-day readmissions are needed. The HOSPITAL score has been internationally validated for medical inpatients, but its performance in select conditions targeted by the Hospital Readmission Reduction Program is unknown.

Design

Retrospective cohort study.

Setting

Six geographically diverse medical centers.

Participants/Exposures

All consecutive adult medical patients discharged alive in 2011 with one of the four medical conditions targeted by the Hospital Readmission Reduction Program (acute myocardial infarction, COPD, pneumonia, and heart failure) were included. Potentially preventable 30-day readmissions were identified using the SQLape algorithm. The HOSPITAL score was calculated for all patients.

Measurements

A multivariable logistic regression model accounting for hospital effects was used to evaluate the accuracy (Brier score), discrimination (C-statistic), and calibration (Pearson goodness-of-fit) of the HOSPITAL score for each four medical conditions.

Results

Among the 9,181 patients included, the overall 30-day potentially preventable readmission rate was 13.6%. Across all four diagnoses, the HOSPITAL score had very good accuracy (Brier score of 0.11), good discrimination (C-statistic of 0.68), and excellent calibration (Hosmer-Lemeshow goodness-of-fit test, P=0.77). Within each diagnosis, performance was similar. In sensitivity analyses, performance was similar for all readmissions (not just potentially preventable) and when restricted to patients age 65 and older.

Conclusions

The HOSPITAL score identifies a high-risk cohort for potentially preventable readmissions in a variety of practice settings, including conditions targeted by the Hospital Readmission Reduction Program. It may be a valuable tool when included in interventions to reduce readmissions within or across these conditions.

Introduction

Under the Affordable Care Act, the Center of Medicare and Medicaid Services (CMS) calculates average risk-adjusted 30-day hospital readmission rates for selected conditions and penalizes hospitals with above-average rates.^1,2 The amount of the penalty and the number of conditions targeted has expanded since the introduction of the Hospital Readmissions Reduction Program (HRRP) in 2012, affecting 78% of hospitals nationally in 2015 with projected total penalties of $428 million.³

Given this expansion, hospitals are seeking to reduce readmissions for penalized conditions. Best practices for doing so include identifying a high-risk cohort, comprehensively assessing potentially modifiable readmission risks, and targeting these risks.^4–7 Progress in this regard has been hindered by lack of a validated tool that could identify potentially preventable readmissions, particularly for conditions targeted by the HRRP. The HOSPITAL score was recently shown to identify patients at risk for potentially preventable readmissions in a cohort of discharged patients across nine hospitals in four different countries.^8,9 However, its performance specifically on selected conditions targeted by the HRRP is unknown. Establishing its performance for each of these conditions is important since hospitals may not face penalties for each condition, and often wish to institute disease-specific interventions to reduce readmissions.¹⁰ If the HOSPITAL score has reasonable performance for these conditions, its international validation and ability to exclude non-preventable readmissions may hold promise for hospitals and clinicians intervening to reduce readmissions. We sought to validate the HOSPITAL score for predicting potentially preventable readmissions within each of the medical conditions targeted by the HRRP, in a multi-site sample of hospital discharges.

Methods

Study Design and Participants

This was a retrospective cohort study of medical inpatients discharged alive between January 1, 2011 and December 31, 2011 from six hospitals in the United States: San Francisco General Hospital, University of California, San Francisco; Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania; Harborview Medical Center at the University of Washington, Seattle, Washington; Vanderbilt University Medical Center in Nashville, Tennessee; Northwestern Memorial Hospital in Chicago, Illinois; Christiana Hospital and Wilmington Hospital in Wilmington, Delaware. Nearly all are large academic tertiary care facilities. All participate in the ICARE (International Cohort of Avoidable REadmissions) consortium.⁹ The non-US hospitals that are a part of ICARE were excluded from this analysis given the focus on conditions targeted by the HRRP specific to the United States. We collected data from the electronic health record (EHR) of individual hospitals because calculation of the HOSPITAL score requires knowledge of laboratory studies, prior admissions, and primary treating service; these factors are not found in national samples of hospital discharges. Since the HOSPITAL score was originally derived and validated in medical inpatients (surgical patients were excluded) this study was similarly restricted to patients discharged from an inpatient medical service.

Patients were excluded if they transferred to another inpatient health care facility (hospital or psychiatric hospital), discharged against medical advice, or had a hospital length of stay of less than 1 day. The study had IRB approval from each site and the managing site at Brigham and Women’s Hospital in Boston, Massachusetts.

Identifying potentially preventable readmissions

All 30-day readmissions to any division of these hospitals were identified first. Since the unit of analysis was a hospital discharge, a discharge could be considered an index admission and a readmission if several hospital stays occurred within 30 days. Potentially preventable readmissions from this cohort were identified using the SQLape algorithm as was done in studies validating the HOSPITAL score.^8,9 This algorithm uses administrative data to categorize readmissions as “unavoidable” or “potentially preventable.”^8,11,12 The SQLape algorithm was developed from physician adjudication of more than 3000 reviewed patient charts, then validated in more than 130,000 patients across 49 hospitals, including a 12-hospital chart review study for validation.¹¹ It uses a similar method to CMS for determining “planned” readmissions, but goes further to identify other unavoidable and avoidable hospitalizations.^13,14 Thus, “potentially preventable” readmissions determined by SQLape would be considered unplanned readmissions by CMS, but not all unplanned readmissions would be considered “potentially preventable” by SQLape. Further details about the SQLape algorithm are listed in the Supplemental Appendix.

As a sensitivity analysis, we also evaluated performance of the HOSPITAL score on all readmissions (not just those considered potentially preventable); in this analysis any patient with a 30-day readmission was compared to those without a readmission in this time frame. This is similar to the CMS measure. We also conducted a sensitivity analysis restricting to patients 65 years of age or older as a proxy for Medicare beneficiary status since we were unable to capture payor source as a part of our dataset and Medicare claims lacked the requisite data to calculate the HOSPITAL score.

Calculating the HOSPITAL score

The development of the HOSPITAL score has been previously described.⁸ Briefly, it was originally derived and internally validated at a single academic US hospital in a sample of discharged medical inpatients.⁸ Logistic regression with backward selection was performed to identify the most significant predictors of potentially preventable readmissions within 30 days of discharge (Table 1). Score points were attributed to each of the 7 predictive variables identified using a regression coefficient-based scoring method. It was then validated in an international sample of all medical inpatients discharged from 9 hospitals from 4 countries (the ICARE consortium, which was created initially for this purpose), involving more than 110,000 records.⁹ This study represents a subsample of this validation study, restricted to the US hospitals that participated.

Table 1.

HOSPITAL score calculation

Characteristics	Value	Points
Low Hemoglobin level at discharge (<12g/dL)	Yes	1
Discharge from an Oncology service	Yes	2
Low Sodium level at discharge (<135mmol/L)	Yes	1
Procedure during hospital stay	Yes	1
Index admission Type: urgent or emergent	Yes	1
Number of hospital Admissions in prior year	≤1	0
	2–5	2
	≥5	5
Index hospitalization Length of stay ≥ 5 days	Yes	2

Legend: The HOSPITAL score has 13 total points as scored above. Bold indicates the letters associated with the HOSPITAL acronym from each characteristic.

The HOSPITAL score (Table 1) includes 7 variably-weighted predictor variables associated with potentially preventable 30-day readmissions in derivation and validation studies.^8,9 These variables include assessment of the number of hospital admissions in the prior twelve months, urgency of admission (urgent or emergent vs. nonemergent), last available laboratory studies (hemoglobin and sodium levels), discharge from an oncology division, any medical procedures or complex imaging performed during the index hospitalization, and the index hospital length of stay. Since only non-surgical patients were enrolled, medical procedures included those traditionally performed by a medical physician, such as endoscopy, hemodialysis, transfusion, percutaneous coronary interventions, or bedside procedures like paracentesis or thoracentesis. Complex imaging studies included computed tomography scans or magnetic resonance imaging. Laboratory studies were the only missing data in this analysis; of 78,921 patients in the cohort, 6259 had missing hemoglobin values, 7.9%, and 3076 had missing sodium values, 3.9%. This means that these laboratory studies were not measured during the course of the hospitalization in question; thus we have assumed that the patient did not have these risk factors.

The scoring system ranges from 0 to 13 points with higher scores connoting higher risk of readmission. These risks were further categorized into three groups: low risk (up to 4 points); intermediate risk (5–6 points); and high risk (7 or more points), roughly corresponding to 5%, 10%, and 20% risk of potentially preventable 30-day readmissions, respectively.^8,9

Measuring HOSPITAL score performance

We used a multivariable logistic regression model including the HOSPITAL score with a fixed effect for hospitals to account for differences in outcomes across hospitals. We employed a robust sandwich variance estimator to account for repeated events by the same patient within hospitals. We calculated the overall performance, the discriminatory power, and the calibration of this model for each of the medical diagnoses targeted by the HRRP: acute myocardial infarction (AMI), chronic obstructive pulmonary disease (COPD), pneumonia, or heart failure (HF). We used Medicare’s methodology for identifying these conditions as described in the 2015 Final Rule.¹⁴ We used the Brier score to quantify accuracy, or how close predictions are to the actual outcome. Scores range from zero (best performance) to 0.25 (non-informative). We calculated the C-statistic to evaluate discrimination; that is, the ability of the HOSPITAL score to discriminate between those with a potentially preventable 30-day readmission and those without a potentially preventable 30-day readmission. The C-statistic was obtained through a logistic regression model that included all variables in the HOSPITAL score, while the calibration evaluates how closely the predicted readmission rate matches the true observed rate within each of the 3 risk score categories described above. Finally, we used the Hosmer-Lemeshow goodness-of-fit test to measure calibration of the model, reporting observed and predicted rates for each deciles of risk; a P ≥ 0.05 implies a good fit, with higher numbers signifying better fit.¹⁶ Analyses were performed with SAS Software, version 9.3 (SAS Institute Inc, Cary, NC).

Results

Of 78,921 discharges during the study period, 9181 (11.6%) were discharged after treatment for one of the HRRP-targeted conditions [pneumonia (n=3,335, 4.2%), heart failure (n=3,189, 4.0%), COPD (n=1,890, 2.4%), acute myocardial infarction (n=767, 1.0%)]. The potentially avoidable readmission rate across these diagnoses was 13.6% overall (Figure 1). Patients treated for HRRP-targeted conditions who had potentially preventable readmissions were more likely to have low sodium and hemoglobin levels and to have more prior admissions in the past year (Table 2).

Figure 1. Cohort description and derivation.

CMS dx = one of the four medical conditions targeted by the Medicare Hospital Readmission Reductions Program. COPD = chronic obstructive pulmonary disease. PAR = potentially avoidable readmission. The number of patients (and proportion of the overall total) are listed.

Table 2.

Baseline characteristics of patients treated for HRRP-targeted condition stratified by 30-day potentially preventable readmission

Characteristics	Potentially preventable readmission N=1252 (13.6%)	No potentially preventable readmission N=7929 (86.4%)	p- value
Age, years, mean (SD)	65.4(15.9)	65.9 (17.0)	0.21
Male (%)	51.4	47.5	0.01
Hospital diagnosis (%)			<0.01
Pneumonia	30.2	37.3
Heart failure	39.2	34.0
COPD	22.8	20.2
Acute MI	7.8	8.5
Low Hemoglobin level	66.5	58.1	<0.01
Discharge from Oncology service	3.8	2.2	<0.01
Low Sodium level	82.2	86.2	<0.01
Procedure during hospital stay	70.2	67.5	0.06
Urgent or emergent Index admissionType	87.9	85.9	0.06
Number of Admissions in past year			<0.01
≤ 1	40.3	66.0
2–5	41.2	29.7
>5	18.5	4.4
Index hospital LOS ≥ 5 days	49.8	42.5	<0.01
HOSPITAL score categories			<0.01
Low risk (≤4 points)	515 (41.1)	4922 (62.1)
Intermediate risk (5–6 points)	347 (27.7)	2027 (25.6)
High risk (≥7 points)	390 (31.2)	980 (12.4)

COPD = chronic obstructive pulmonary disease, MI = myocardial infarction, LOS = length of stay.

Across all four diagnoses, the HOSPITAL score had very good accuracy (Brier score=0.11) good discrimination [C-statistic = 0.68 (95%CI 0.66–0.70)], and very good calibration (Hosmer-Lemeshow goodness-of-fit P value=0.77). For example, the expected and observed readmission rates were very similar within each risk subgroup: low risk (9.1% expected, 9.6% observed), moderate risk (11.3% expected, 11.0% observed), and high risk (18.0% expected, 18.1% observed). Within diagnoses, accuracy and discrimination were similar (Brier score 0.10–0.12; c-statistic 0.67–0.71), though calibration was better for pneumonia and COPD (P = 0.76 and 0.81, respectively) than for acute MI or heart failure (P=0.16 and 0.17, respectively – Table 3a). Results were similar when evaluating only patients age 65 and older (Table 3b) and when considering all readmissions (not just those considered potentially preventable – Table 3c).

Table 3.

a. HOSPITAL Score performance in potentially preventable 30-day readmissions
	Discharges (N)	30-day readmission rate	Brier score	C-statistic (95%CI)	Hosmer- Lemeshow (P-value)
All CMS diagnoses	9,181	13.6%	0.11	0.68 (0.66–0.70)	0.77
Pneumonia	3,335	11.3%	0.10	0.68 (0.65–0.71)	0.76
Heart failure	3,189	15.4%	0.12	0.68 (0.65–0.70)	0.17
Acute MI	767	12.7%	0.11	0.67 (0.61–0.72)	0.16
COPD	1,890	15.1%	0.12	0.71 (0.67–0.74)	0.81

b. HOSPITAL Score performance in potentially preventable 30-day readmissions restricted to patients age 65 years or older
	Discharges (N)	30-day readmission rate	Brier score	C-statistic (95%CI)	Hosmer- Lemeshow (P-value)
All CMS diagnoses	5,087	13.2%	0.11	0.66 (0.63–0.68)	0.27
Pneumonia	1,717	11.8%	0.10	0.64 (0.60–0.69)	0.27
Heart failure	1,918	14.1%	0.12	0.68 (0.64–0.71)	0.37
Acute MI	421	12.4%	0.12	0.68 (0.61–0.76)	0.56
COPD	1,031	14.1%	0.10	0.67 (0.62–0.72)	0.71

c. HOSPITAL score performance in all 30-day readmissions
	Discharges (N)	30-day readmission rate	Brier score	C-statistic (95%CI)	Hosmer- Lemeshow (P-value)
All CMS diagnoses	9,181	16.3%	0.13	0.68 (0.66–0.69)	0.41
Pneumonia	3,335	13.8%	0.13	0.68 (0.65–0.70)	0.49
Heart failure	3,189	18.2%	0.13	0.67 (0.65–0.70)	0.10
Acute MI	767	17.4%	0.13	0.66 (0.61–0.71)	0.49
COPD	1,890	16.6%	0.13	0.70 (0.67–0.73)	0.84

Discussion

In this multicenter retrospective cohort, the HOSPITAL score had very good accuracy, discrimination, and calibration for 30-day potentially preventable readmissions after inpatient treatment for one of the four medical conditions targeted by the Hospital Readmission Reduction Program. These findings are important for practitioners and hospitals seeking to reduce preventable readmissions and improve patient outcomes.

The HOSPITAL score may fill a needed niche as a risk prediction score that is validated in multiple institutions in the United States and abroad, and identifies a high-risk population for potentially avoidable admissions. The HOSPITAL score has better performance characteristics than most prediction models using retrospective data in all-cause readmissions and within specific disease states, separating the HOSPITAL score from prior risk prediction models.¹⁷ In particular, our results suggest that the HOSPITAL score can be used both to support comprehensive transitional care interventions which have a track record of success,^18,19 or disease-specific interventions within diagnoses penalized as a part of the HRRP. Examples of integrating a risk prediction score into an intervention are preliminary,²⁰ but suggest such a model tailors the intervention to those most at risk, offering the most promise of reducing readmissions in a maximally cost-effective manner.

Although the HOSPITAL score can help predict a high-risk cohort for readmission and segregates out likely non-preventable readmissions, it does not explain why the patient might be readmitted or how to intervene to reduce readmission risk. Many of the markers in the score (such as previous hospital admissions) are clearly not modifiable, and others (low hemoglobin) are likely associated with sicker patients rather than representing targets for intervention. Rather, the score could be used to identify a high-risk cohort, who then could be formally assessed for risk factors for readmission that might be modifiable, and receive a tailored intervention.^4,6 Surveys of hospital administrators indicate they use both disease-specific and more general interventions to reduce readmissions.^21–23 A relative weakness of the HOSPITAL score in this regard is that it requires data available shortly before discharge (length of stay and laboratory studies) which may ultimately affect which risk group the patient fits into. However, clinicians may find that the length of stay and likely ranges for these laboratory studies can often be predicted within 1–2 days prior to discharge.

The proportion of HRRP-targeted conditions in our sample was similar to other national data.²⁴ Despite targeting a minority of conditions, readmission rates may be improving nationally both for targeted and non-targeted conditions^24–26; Medicare is adding additional conditions each year.¹⁵

While a full discussion of the relative preventability of readmissions is beyond the scope of this paper,^27–29 we note recent studies assessing preventability in the eye of patients and providers document high rates of potentially preventable readmissions, though these still constitute a minority of readmissions overall.^30–32 In our cohort, the preventable readmission rate was lower than in these studies. This may reflect the difference between using administrative data versus interviews or comprehensive chart review of readmissions to determine preventability. It should be noted that there is currently no consensus on how to identify with certainty which readmissions are truly preventable, no matter how laborious the process. Not all potentially preventable readmissions identified through administrative data and/or EHRs (such as those identified by SQLape) may be preventable. For example, in SQLape’s largest validation study, the true positive rate for identifying potentially preventable readmissions was 78% based on medical chart review, with most errors related to coding rather than errors in the algorithm.¹¹ However, we feel the advantages of SQLape – including its transparent design, large multisite derivation and validation samples with chart review substudies for validation – outweigh potential misclassification bias, particularly compared to the CMS all-cause readmission measure (which does not attempt to identify any readmissions as potentially preventable) or other tools to assess preventable readmissions using administrative data, which have not been externally validated.³³ Many hospitals and providers will seek to identify an enriched cohort of potentially avoidable readmissions using administrative data, a niche the HOSPITAL score may help to fill.

This study should be considered in the context of the data analyzed. Since the HOSPITAL score was derived and validated in medical patients, it has not been studied in surgical populations or for surgical diagnoses (e.g., joint replacement) targeted by the HRRP. While very few patients were coded as being discharged from an oncology service, none of the HRRP-targeted measures is primarily associated with an oncologic diagnosis, raising concern about whether this coding was accurate. Anecdotally, oncology services at some hospitals in our sample also admit their own patients for acute medical concerns, not just for scheduled chemotherapy, potentially explaining this finding.

The HOSPITAL score does not include several important predictors of hospital readmission, such as functional status,^32,33 socioeconomic status,³⁴ or hospital characteristics.²⁵ However, these characteristics are infrequently captured in administrative data, and the HOSPITAL score was purposefully limited to seven variables to increase clinical applicability and ease of use. Diagnoses treated are subject to coding error, and we were unable to conduct chart review to confirm coding accuracy or validate readmissions flagged as potentially avoidable using the SQLape algorithm. Although the original score was validated in nine centers across four countries, this analysis was limited to predominantly large academic medical centers, and the predictive characteristics of the HOSPITAL score may be different in other settings for HRRP-targeted conditions.

We did not compare the performance of the HOSPITAL score with CMS’s risk-adjustment model for hospital readmissions since the intent of the two models is different. The HOSPITAL score was derived and validated to predict 30-day potentially avoidable readmissions, while CMS’s risk-adjustment measure was developed to enable comparison of readmission rates across hospitals through adjustment for patient readmission risk factors that may be unequally distributed across hospitals. We also did not attempt to derive a new risk prediction score for HRRP-targeted conditions, but rather to analyze the performance of the original HOSPITAL score in these specific populations. Thus, while some predictors that were valid in the original derivation study did not meet statistical significance and other predictors could have better performance in these select populations, the overall value of the HOSPITAL score lies in its prediction of potentially preventable readmissions across medical inpatients in a variety of contexts, as well as its performance for HRRP-targeted conditions. A single score that performs well in all contexts has more utility than multiple individualized scores specialized to specific contexts.

The HOSPITAL score is a valid risk prediction model for 30-day potentially preventable readmissions across all readmissions and within important disease-specific categories. As such, it may serve as a key part of future cost-effective interventions to reduce preventable readmissions.