Reliability of Predicting Early Hospital Readmission After Discharge For An Acute Coronary Syndrome using Claims-Based Data

Abstract

Early rehospitalization after discharge for an acute coronary syndrome (ACS), including acute myocardial infarction (AMI), is generally considered undesirable. The Centers for Medicare and Medicaid Services (CMS) base hospital financial incentives on risk-adjusted readmission rates following AMI, using claims data in its adjustment models. Little is known about the contribution to readmission risk of factors not captured by claims. For 804 consecutive patients over 65 years old discharged in 2011–13 from 6 hospitals in Massachusetts and Georgia after an ACS, we compared a CMS-like readmission prediction model with an enhanced model incorporating additional clinical, psychosocial, and sociodemographic characteristics, after principal components analysis. Mean age was 73 years, 38% were women, 25% college educated, 32% had a prior AMI; all-cause re-hospitalization occurred within 30 days for 13%. In the enhanced model, prior coronary intervention [Odds Ratio=2.05 95% Confidence Interval (1.34, 3.16)], chronic kidney disease [1.89 (1.15, 3.10)], low health literacy [1.75 (1.14, 2.69)], lower serum sodium levels, and current non-smoker status were positively associated with readmission. The discriminative ability of the enhanced vs. the claims-based model was higher without evidence of over-fitting. For example, for patients in the highest deciles of readmission likelihood, observed readmissions occurred in 24% for the claims-based model and 33% for the enhanced model. In conclusion, readmission may be influenced by measurable factors not in CMS’ claims-based models and not controllable by hospitals. Incorporating additional factors into risk-adjusted readmission models may improve their accuracy and validity for use as indicators of hospital quality.

Introduction

Early readmissions are often considered preventable and reflective of poor in-hospital management or discharge practices.¹ Eight years ago, the Medicare Payment Advisory Commission called for the development of new measures to aid in predicting readmission after myocardial infarction, which, along with six other conditions, was thought to contribute to nearly one third of potentially preventable early readmissions. In 2011, a claims-based method developed by Krumholz et al. was approved by the National Quality Forum and implemented by the Centers for Medicare and Medicaid Services (CMS) for estimating hospitals’ risk-standardized readmission rates for patients discharged following an AMI.² Subsequently, the Affordable Care Act established the Hospital Readmission Reduction Program and payment reforms that penalize hospitals with higher than predicted readmission rates. These penalties heighten interest in identifying and improving transitional care for ACS patients at high risk for readmission.^3,4 Despite the high economic and clinical stakes, little is known about the impact of clinical, psychosocial, and sociodemographic factors on rehospitalization in patients admitted with an ACS.^1,5,6 The claims-based model of Krumholz et al. is only modestly discriminating. We hypothesized that adding patient clinical, psychosocial, and sociodemographic information could improve the performance of this readmission prediction model. Therefore, using data from a cohort of Medicare-age patients discharged after an ACS, we compared the performance of a CMS-like model to each of 3 models that incorporated a number of variables representing clinical, psychosocial, and sociodemographic characteristics, respectively.

Methods

Details of the design, participant recruitment, interview processes, and medical record abstraction procedures used in TRACE-CORE (Transitions, Risks and Action in Coronary Events) have been previously described.^7,8 In brief, TRACE-CORE used a 6-site prospective cohort design to follow 2,187 adults discharged alive after An ACS hospitalization. Participants with an ACS were identified using active surveillance methods by trained study staff between April 2011 and May 2013. Adult patients admitted to any of the 6 participating medical centers with electrocardiographic or cardiac biomarker criteria consistent with an ACS, those undergoing urgent coronary revascularization, and symptomatic participants with >70% stenosis in a coronary artery on coronary angiography were considered eligible. Pregnant patients, patients with dementia or receiving palliative care, patients with an ACS secondary to demand ischemia, perioperative ACS cases, and patients under custody of a prison system were ineligible. The 6 participating hospitals were selected for their diverse patient population; also, the catchment areas of these 6 hospitals are such that, if early readmission occurred, it was very likely to be at one of the study hospitals. Sites included 2 academic teaching hospitals and a large community hospital that cover essentially all ACS admissions in Central Massachusetts. The other sites included 2 hospitals affiliated with a managed care organization in Atlanta, GA and a tertiary care academic medical center covering most ACS admissions in Central GA. The Institutional Review Boards at each participating recruitment site approved this study. All participants provided written informed consent.

For comparability with the dataset used to generate and test the original CMS model, we excluded TRACE-CORE participants younger than 65 years (n=1321), those with planned readmissions (n=30), and patients who died within 30 days of discharge (n=7). We also excluded patients with missing data during the index hospitalization (n=13), and those participants who were same-day discharges (n=12, Supplemental Appendix A), resulting in an analytic sample of 804 elderly adults discharged following ACS (Supplemental Appendix A).

Trained study staff abstracted participants’ baseline demographic, clinical, laboratory, and electrocardiographic data as well as in-hospital clinical complications from available hospital medical records. Comorbidities present at the time of hospital admission were identified from each participant’s admission history and physical examination. Development of complications during hospitalization were defined according to validated criteria.⁹ We re-abstracted 5% of randomly selected charts to confirm high inter-rater reliability at all sites.⁸ Clinical data were used to derive the GRACE risk score.¹⁰ Each participant’s discharge summary was reviewed to confirm an ACS diagnosis and to characterize it as unstable angina, non-ST-segment elevation myocardial infarction (NSTEMI), or ST-segment elevation myocardial infarction (STEMI) based on established criteria.^10–13 Questionable cases were adjudicated by 2 study physicians blinded to clinical diagnosis.

Trained interviewers conducted a computer-assisted face-to-face interview during each participant’s index hospitalization for ACS or by phone within 72 hours of discharge. We assessed cognitive impairment using the Telephone Interview for Cognitive Status (TICS).¹⁴ To assess severity of depressive symptoms, participants completed the Patient Health Questionnaire (PHQ-9).¹⁵ Study patients also completed the Generalized Anxiety Disorder (GAD-7) questionnaire,¹⁶ a validated scale. Participants completed the 4-item version of the Perceived Stress Scale, a validated measure of the degree to which situations in one’s life are seen as stressful.¹⁷ To assess participants’ engagement in their health care we included the Patient Activation Measure (PAM6).¹⁸ Participants also completed brief screens for low health literacy and numeracy.^19,20 We also included 6 questions about social support from the Medical Outcomes Study Social Support Survey,²¹ alcohol use from the AUDIT-C questionnaire, and assessed smoking status and use of smokeless tobacco products using items from the TRIUMPH study.²²

On subsequent structured follow-up interviews,¹⁴ participants reported emergency visits and hospital readmissions during follow-up telephone interviews. Post-discharge records were reviewed to confirm the patient’s readmission status and provide data on the timing and reason for rehospitalization. For the present study, a hospital readmission was considered present only if confirmed using medical record data. All-cause mortality was ascertained from proxy reports and review of medical records augmented by review of local and national vital statistics records.

All study participants were discharged alive following an ACS hospitalization. Our primary study outcome was whether the patient had an unscheduled readmission at any of our 6 participating hospitals for any reason during the following 30 days. In this paper we use the term “readmission” to indicate any unplanned rehospitalization within 30 days of discharge from the index ACS admission. We examined the relationship between readmission and several pre-existing factors included in the CMS model,² additional in-hospital clinical factors, psychosocial, sociodemographic factors, and in-hospital complications. We used analysis of variance and chi-squared statistics to test differences in individual socio-demographic, psychosocial, clinical, and treatment factors between those with and without readmission.

Because there were only 106 readmissions among our 804 participants, we needed to limit the number of variables used for prediction. Thus, within each block of variables representing a distinct domain, we first conducted a principal components analysis and then chose the number of high-information summary variables that had the lowest AIC (Akaike Information Criterion) when used to predict readmission. Block 1 included abstracted information from 18 variables in CMS’ model that are available in TRACE-CORE. We did not have the data to include the following additional predictors that the CMS model extracts from claims data: asthma, urinary tract infection, pneumonia, metastatic cancer, hemiplegia, chronic skin ulcer, malnutrition, any infection, and electrolyte or fluid disorder. In Model 2, based on a priori assumptions and known relations between these factors and survival,¹ we studied the additional explanatory power associated with clinical factors not currently considered by CMS (Block 2 added to Block 1). In Model 3 (Blocks 1–3), we added several mental health-related factors and cognition (Block 3) to Model 2. In Model 4 (Blocks 1–4), we added information on sociodemographic and additional psychosocial characteristics (Block 4) to Model 3.

The first eigenvector for Block 1 (CMS-like variables) retained essentially all of the true explanatory power of the 18 individual predictors (Table 1, Supplemental Appendix B). In subsequent models, this single factor was used to represent the Block 1 variables. In Block 2, a single eigenvector captured most of the total additional explanatory power of the clinical block (Table 2, Supplemental Appendix B) and was used to represent the block in further modeling. In Block 3, again, a single eigenvector accounted for the majority of the variance among the mental health factors (Table 3, Supplemental Appendix B). One factor from Block 4 emerged (Table 4, Supplemental Appendix B) as important, and was retained, giving rise to the final enhanced model. We also considered data reduction for Block 5 variables (major in-hospital complications), and initially retained one such factor (Table 5, Supplemental Appendix B). Although we initially studied 5 sequential models, the model that included in-hospital complications performed no better than the previous model that included the variables represented in the other 4 blocks, so our final enhanced model was Model 4, which included the base model variables plus clinical, mental health and cognition, psychosocial and socio-demographic characteristics. All model comparisons were performed using these dimension-reduced factors to represent each block. We compared model discrimination using the C statistic. For each model, we assessed calibration using the Hosmer-Lemeshow chi-squared statistic. Better fitting models have smaller chi-squared values. We also provide an accessible measure of discrimination by examining the difference in readmission rates among subjects in the lowest and highest deciles of predicted risk. For each model, we used box plots to characterize the distributions of predicted risk, separately among those with and without readmission; we also calculated readmission rates within the lowest and highest deciles of predicted risk. Better models have lower risk scores for those without readmission, higher scores for those readmitted and more extreme differences between the readmission rates of those in the lowest and highest deciles of predicted risk. All analyses were performed using SAS version 9.3 (Cary, NC).

Results

Participants were on average 73 years old, 38% were women, and 1 in 4 were college educated (Table 1). There was a high burden of comorbid illnesses. The mean GRACE risk score was 117, suggesting intermediate risk for in-hospital death.²³ The average length of hospital stay was 4.6 days (median 3 days). Slightly more than half of participants experienced a NSTEMI, and 65% received a percutaneous coronary intervention during their index ACS admission.

Table 1.

Characteristics of patients discharged after an acute coronary syndrome hospitalization by 30 day-readmission status: TRACE-CORE, 2011–2013

Characteristic	Readmission		P-value
	No (N = 698)	Yes (N = 106)
Block 1 – CMS Factors
Age (mean ± SD, years)	72.4 ± 5.9	74.0 ± 6.5	0.024
Male	425 (60.9%)	70 (66.0%)	0.31
Percutaneous coronary intervention	209 (29.9%)	50 (47.2%)	0.0006
Coronary artery bypass	167 (23.9%)	27 (25.5%)	0.73
Coronary artery disease or acute myocardial infarction	390 (55.9%)	74 (69.8%)	0.006
Heart failure	116 (16.6%)	23 (21.7%)	0.21
Atrial fibrillation	96 (13.8%)	17 (16.0%)	0.53
Valvular heart disease	33 (4.7%)	7 (6.6%)	0.43
TIA/Stroke	79 (11.3%)	21 (19.8%)	0.020
Peripheral vascular disease	76 (10.9%)	17 (16.0%)	0.14
Diabetes mellitus	237 (34.0%)	37 (34.9%)	0.85
Chronic kidney Disease	100 (14.3%)	30 (28.3%)	0.0006
Dialysis	10 (1.4%)	4 (3.8%)	0.13
Chronic lung disease	147 (21.1%)	25 (23.6%)	0.56
Anemia	49 (7.0%)	12 (11.3%)	0.14
Alzheimer’s Disease	2 (0.3%)	1 (0.9%)	0.37
Cancer	116 (16.6%)	26 (24.5%)	0.06
Hypertension	582 (83.4%)	96 (90.6%)	0.045
Presenting characteristics
Anterior myocardial infarction	47 (6.7%)	3 (2.8%)	0.09
Block 2 – Clinical Factors
ACS Type
STEMI	102 (14.7%)	14 (13.3%)	0.99
NSTEMI	359 (51.8%)	57 (54.3%)
UA	232 (33.5%)	34 (32.4%)
Physiologic and laboratory characteristics: mean ± SD
Systolic BP (mmHg)	141.7 ± 25.2	143.6 ± 26.7	0.48
Pulse (bpm)	75.7 ± 18.8	77.1 ± 15.2	0.39
Respiratory rate (rpm)	18.2 ± 3.5	18.4 ± 3.3	0.56
Creatinine (mg/dL)	1.4 ± 1.0	1.7 ± 1.8	0.07
Max troponin I (mg/dL)	14.7 ± 28.3	18.5 ± 40.2	0.43
Min sodium (mg/dL)	135.8 ± 3.4	134.6 ± 3.8	0.0037
Min hemoglobin (mg/dL)	11.2 ± 2.2	10.7 ± 2.5	0.044
Block 3 – Mental health Factors and Cognition
Severe Depression*	12 (1.7%)	2 (1.9%)	0.90
Severe Anxiety**	39 (5.6%)	11 (10.4%)	0.08
High Perceived Stress***	214 (30.7%)	44 (41.2%)	0.029
Cognitively Impaired****	163 (23.4%)	30 (28.3%)	0.27
Block 4 – Socioeconomic and Psychosocial Factors
Education
High School or Less	322 (46.1%)	48 (45.7%)	0.95
Some college	203 (29.1%)	32 (30.5%)
College graduate	173 (24.8%)	25 (23.8%)
Non-Hispanic White	611 (88.0%)	91 (85.9%)	0.74
Non-Hispanic Black	62 (8.9%)	12 (11.3%)
Hispanic	21 (3.0%)	3 (2.8%)
Low Social Support*****	25 (3.6%)	6 (5.7%)	0.33
Lives Alone	176 (25.2%)	34 (32.1%)	0.14
Low Health Literacy†	247 (35.4%)	53 (50.0%)	0.0053
Low Health Numeracy††	411 (58.9%)	57 (53.8%)	0.32
Heavy Alcohol Drinker	42 (6.0%)	4 (3.8%)	0.33
Ever Smoker	487 (69.8%)	66 (62.3%)	0.13
Block 5 – In-Hospital Complications
In-hospital clinical complications
Cardiac Arrest	5 (0.7%)	1 (0.9%)	0.81
Cardiogenic Shock	7 (1.0%)	2 (1.9%)	0.46
Heart Failure	19 (2.7%)	3 (2.8%)	0.95

^*PHQ-9 Patient Health Questionnaire 9 item score (5–9 mild; 10–14 moderate; 15–19 moderately severe; and ≥ 20 severe depression);

^**GAD-7 General Anxiety Disorder 7 item score (5–9 mild; 10–14 moderate; ≥15 Severe);

^***Perceived Stress Scale score (≥20 high perceived stress);

^****TICS – telephone interview for cognitive status score (≤28 impaired);

^*****Medical Outcomes Survey: Social Support Score (<12 low support);

^†Somewhat/Not at all confident/little confidence in filling out medical form;

^††Unable to answer both health numeracy questions correctly

There were 106 documented readmissions at our study hospitals within 30 days of discharge. Being older and having a previous history of coronary artery disease and several important CVD comorbidities were associated with readmission (Table 1). Major in-hospital complications, which occurred infrequently in survivors of the index hospitalization, were not associated with 30-day readmission. In the final enhanced model, receipt of a prior coronary intervention and history of chronic kidney disease were the only factors from the CMS model significantly associated with readmission (Table 2). Regarding additional factors included in Model 2, lower mean minimum sodium values were associated with a higher risk and current smoking lower risk, of early readmission (Table 2). Of the psychosocial and socioeconomic characteristics included in Models 3 and 4, low health literacy was associated with 75% increase in the odds of early readmission (Table 2).

Table 2.

Crude and Adjusted Odds Ratios of Readmission for Variables Associated with Readmission within 30 days: TRACE-CORE, 2011–2013

Factor Associated with Readmission	Crude Odds Ratio	95% Confidence Interval	Multivariable Adjusted* Odds Ratio	95% Confidence Interval
Prior Coronary Intervention	2.09	1.38, 3.16	2.05	1.34, 3.16
Prior Chronic Kidney Disease	2.36	1.47, 3.79	1.89	1.15, 3.10
Minimum Serum Sodium**	0.90	0.86, 0.95	0.91	0.86, 0.97
Current Smoking	0.31	0.11, 0.87	0.25	0.09, 0.72
Low Health Literacy	1.83	1.21, 2.75	1.75	1.14, 2.69

^*From the enhanced model adjusting for all factors associated with readmission with a p <0.1.

^**Each 1 mEg/L higher in minimum serum sodium (mEq/L) was associated with a 9% lower adjusted odds of 30-day readmission.

As shown in Table 3, the CMS-based Block, which relies on a single factor/reduced version of the Krumholz et al. instrument used by CMS for readmission risk stratification,² achieved a C-statistic of 0.63. Adding the first principal component from the clinical factor block to the base CMS Block (Block 2) moderately improved predictive performance, without over-fitting, since the augmented model had a lower AIC score. Adding the principal components from the mental health block as well as sociodemographic - psychosocial block further enhanced the accuracy of the composite model (Table 3), as demonstrated by a higher C statistic observed with each added domain, without significant over-fitting. In contrast, the addition of the principal component derived from Block 5 did not improve the accuracy of the predictive instrument and resulted in a higher AIC (Table 3).

Table 3.

Performance Characteristics of the 30-Day Readmission Prediction Models: TRACE-CORE, 2011–2013

	DF	C statistic	AIC*
Base Model (Model 1)	1	0.629	614.17
Model 2	2	0.645	607.52
Model 3	3	0.653	608.05
Model 4	4	0.653	606.45

Base Model = CMS-like model including factors from Block 1 in Table 1. Model 2 = base model plus clinical factors in Block 2. Model 3 = Model 2 plus mental health characteristics from Block 3. Model 4 = Model 3 plus additional psychosocial and sociodemographic characteristics from Block 4. AIC = Akaike Information Criterion. DF = model degrees of freedom (that is, the number of variables in the model)

Figure 1 depicts the predicted versus observed readmission numbers, by decile of predicted risk, for the base model (claims-based) and for the enhanced model (Model 4, or the base CMS model plus clinical, mental health and cognition, psychosocial and socio-demographic characteristics). TRACE-CORE participants in the highest decile of predicted risk based on the enhanced model had the highest rate of observed readmissions (31.3% vs. 23.8% in the base CMS model). On the basis of the Hosmer-Lemeshow chi-square test, all 4 models had adequate calibration. TRACE-CORE participants in the highest decile of predicted risk based on Model 4 had the highest rate of observed readmissions (32.5% vs. 23.7% in the base CMS model). On the basis of the Hosmer-Lemeshow chi-square test, all models had adequate calibration.

Figure 1.

Predicted and Observed Numbers of Readmissions by Decile of Predicted Readmission Likelihood According to Claims-Based and an Enhanced Model: TRACE-CORE, 2011–2013.

DISCUSSION

In this multi-center study of over 800 Medicare-age patients discharged from the hospital after an ACS between 2011 and 2013, all-cause readmissions occurred in approximately 1 of every 8 patients within 30 days of hospital discharge. We observed that chronic coronary or kidney disease, lower minimum serum sodium values, current non-smoking status, and low health literacy during the index hospitalization were each associated with early readmission. In contrast to studies examining mortality as an outcome, we saw no association between ACS type or in-hospital complications with early readmission.²³ Adding clinical and psychosocial patient characteristics that are not usually assessed when measuring quality of care to a claims-based model enhanced prediction of readmission.

Despite restricting our sample to elderly patients and the increasing risk of readmission with advancing age, the rates of 30 day rehospitalization observed in our contemporary patient cohort were at the lower end of previously-reported rates (11–30%).¹ As reported in a recent systematic review including >35 studies and >120,000 patients discharged from the hospital after an ACS, few demographic characteristics or cardiovascular conditions have shown consistent associations with early hospital readmission.^24,25 Consistent with prior studies, we observed associations between age and history of hypertension, coronary disease, stroke and chronic kidney disease with early readmission on univariate testing. However, of the candidate clinical factors previously examined, only prior cardiac intervention and kidney disease showed statistically significant relations to early hospital readmission after multivariable adjustment.²⁴

Surprisingly, lower minimum serum sodium levels measured during the index hospitalization were associated with increased risk of hospital readmission. Although serum sodium is a known predictor of readmission in patients admitted with acute decompensated heart failure,²⁶ we found no prior study examining serum sodium levels and readmission following an ACS hospitalization. It may also be that sodium levels are a marker for other clinical morbidity or vulnerability that was not captured in the information available in the current study. Being a current smoker was inversely associated with readmission, a relationship not previously reported, to our knowledge. We speculate that there may be residual confounding by healthcare seeing behavior, general health status, survivor bias, age, and comorbidity in the smoker-readmission association we observed.

Although such factors have rarely been examined in relation to readmission, it is plausible that sociodemographic and psychosocial characteristics, including education level, social support, health-related knowledge, and depression, contribute to readmissions.²⁷ In fact, several studies have found increased readmissions to be associated with depression, lower education, greater stress, lesser social support, and lower health-related knowledge.^27,28,29 Although the only psychosocial or sociodemographic characteristic associated with readmission in this study was low health literacy, the exclusion of patients with delirium, dementia, and severe cognitive dysfunction may have led to underestimation of the influence of these psychosocial factors on hospital readmission.

Our finding that participants with low health literacy were at greater risk of unplanned readmissions within 30 days warrants attention. We assessed health literacy with a single question that has been validated previously.¹⁹ This brief measure would be easy to administer during hospitalization and might identify patients with additional care transitions needs or follow-up to prevent unplanned readmissions. Clinicians may also want to explore different ways of communicating important discharge information to patients with impaired health literacy, with the aim of improving comprehension and thus reducing readmissions.³⁰

A recent meta-analysis of readmission risk prediction instruments found that, despite ongoing use of such models by policy-makers and clinicians, readmission models have relatively poor predictive ability. We observed comparable performance of a CMS-like model in our sample to that observed by Krumholz et al. in their seminal work.² The modest performance of existing readmission prediction models highlights the fact that risk models including factors available in administrative claims data are generally much better suited for predicting survival rather than hospital readmission.²³ In fact, many patient-level factors associated with poorer post-discharge survival, including adverse health behaviors, may be associated with lower rates of readmission. We conjecture that these behaviors reflect lower ability to attend to personal well-being and lower likelihood of seeking care for a health conditions but our findings may also be explained by survivor bias.

We observed that adding information about clinical, mental health, and socio-demographic characteristics could improve the C-statistic of a CMS-like early readmission prediction model, without overfitting. The claims-based model currently employed by CMS for predicting readmission appears to not fully account for differences in patient risk. Factors that are not under a hospital system’s control, but are associated with readmission, such as low health literacy, are not presently accounted for when readmissions are risk-adjusted and the adjusted rate is used as a measure of the quality of care. Although we realize that no model can be perfect, contemporary hospital systems are graded and increasingly being penalized for poor performance compared to risk-adjusted readmission benchmarks.⁶ Given that some hospitals – for example, safety net hospitals – see disproportionate numbers of persons with non-medical vulnerabilities not currently adjusted for, our study suggests that models incorporating additional dimension of risk might be more equitable.

Our study has several limitations. In constructing the predictive models, we did not have data on all factors included in the full model used by CMS and our ability to identify individual factors of importance was limited by small numbers. The CMS-like model did, however, include the majority of key administrative-level data elements and performed similarly in our cohort to the full model as it was developed and tested in other cohorts.