Predicting 30-day mortality and 30-day re-hospitalization risks in Medicare patients with heart failure discharged to skilled nursing facilities: development and validation of models using administrative data

Abstract

Background:

Despite the growing importance of skilled nursing facility care for Medicare patients hospitalized with heart failure, no risk prediction models for these patients exist.

Objectives:

To develop and validate separate predictive models for 30-day all-cause mortality and 30-day all-cause re-hospitalization

Design:

Retrospective cohort study using a nationwide Medicare claims data cross-linked with Minimum Data Set 3.0

Setting:

11,529 skilled nursing facilities in the United States (2011–2013)

Participants:

77,670 hospitalized heart failure patients discharged to skilled nursing facilities (randomly split into development (2/3) and validation (1/3) cohorts).

Measurements:

Using data on patient sociodemographic and clinical characteristics, health service use, functional status, and facility-level factors, we developed separate prediction models for 30-day mortality and 30-day re-hospitalization using logistic regression models in the development cohort.

Results:

Within 30 days, 6.8% died and 24.2% were re-hospitalized. Thirteen patient-level factors remained in the final model for 30-day mortality and 10 patient-level factors for re-hospitalization with good calibration. The area under receiver operating characteristic curves were 0.71 for 30-day mortality and 0.63 for re-hospitalization in the validation cohort.

Conclusions:

Among Medicare patients with heart failure discharged to skilled nursing facilities, predicting 30-day mortality and re-hospitalization using administrative data is challenging. Further work identifying factors for re-hospitalization remains needed.

Introduction

Heart failure (HF) currently affects ~6.5 million American adults,¹ and is the leading cause of hospitalization among persons aged ≥65 years.² Nearly one-quarter of Medicare beneficiaries are discharged to skilled nursing facilities (SNFs) after being hospitalized for HF.^{3, 4} These patients are typically older, and have greater mortality and re-hospitalization risks compared with those discharged to home.⁴ SNFs play a critical role in the care continuum of patients with HF, from transitions from the hospital to the SNF and then from SNFs back to patients’ residences.⁵ The SNF care use for patients with HF has steadily increased in recent decades, which may be partially due to changes in Medicare reimbursement policies.⁶

Little is known about factors that predict short-term mortality and re-hospitalization in the high-risk SNF patients. Studies have examined the risks of dying or being re-hospitalized among patients with HF;^7–11 no models exist specifically for patients with HF discharged to SNFs.¹² Older patients with HF are particularly vulnerable during the 30-day period after being discharged from the hospital after an episode of HF.¹³ Understanding predictors of 30-day outcomes at the time of admission to a SNF could guide the intensity of clinical monitoring, frequency of patient reassessments, and the extent of clinical intervention needed.¹² We developed and internally validated predictive models for 30-day mortality and re-hospitalization risks.

Methods

Medicare Provider Analysis and Review (MedPAR) files were linked to Minimum Data Set (MDS) 3.0, Provider of Services files, Nursing Home Compare Data, and the Medicare Beneficiary Summary Files (MBSF) for 2011–2013. MedPAR includes administrative and clinical elements obtained from discharge abstracts for hospitalizations and SNF stays of fee-for-service beneficiaries. The MDS is a federally-mandated comprehensive clinical assessment of residents living in all Medicare/Medicaid certified nursing facilities. It captures resident-level items including demographic characteristics, functional status, and diagnoses. For SNF residents, the standard Centers for Medicare and Medicaid Services (CMS) assessment occurs on days 5, 14, 30, 60, and 90.¹⁴ The reliability and validity of MDS 3.0 items including residents’ medical, cognitive, functional, and psychological status has been confirmed.^{15, 16} The MBSF contains beneficiaries’ demographic, enrollment, and death related information. The University of Massachusetts Medical School Institutional Review Board approved the study.

We identified all episodes of an acute hospitalization with ≥3 consecutive days of stay¹⁷ with a primary diagnosis of HF¹⁸ discharged to a SNF on the day of or day after the index hospital discharge (Appendix Figure 1). For those with multiple eligible episodes, one was randomly selected.

Andersen’s behavioral model of health services¹⁹ guided the selection of potential patient-level predictor variables with a focus on predisposing and need factors. Predisposing factors included patient’s age, gender, race/ethnicity (non-Hispanic white, non-Hispanic black, Hispanic, or non-Hispanic other), marital status (married, widowed, separated/divorced, or never married), and dual eligibility status (Medicare and Medicaid dual enrollment or Medicare only). Need factors were comorbidities, health service use, health characteristics, and functional status. We considered SNF characteristics in our modeling approaches because facility associated factors (e.g.,nurse staffing) are associated with outcomes of care.¹² Potential predictors with <5% prevalence or ≥10% missingness were excluded.

Comorbidities included 2 acute (pneumonia and urinary tract infection based on the MDS) and 11 chronic diagnoses (coronary artery disease, atrial fibrillation, cerebrovascular disease, peripheral vascular disease, hyperlipidemia, diabetes, chronic obstructive pulmonary disease (COPD) or asthma, anemia, chronic kidney disease (stage ≥3), dementia, and depression in 6 months before index date). We calculated a Charlson index.²⁰ We classified patients as having systolic, diastolic, and unspecified HF based on the primary diagnosis at the time of the index hospitalization.¹⁸

Health service use need factors were based on acute hospitalization claims: length of the index hospital stay, stay in an intensive care unit during the patient’s index hospitalization (yes/no), number of hospitalizations in 6 months before the index hospitalization (0, 1, ≥2), number of hospitalizations for HF (0, 1, ≥2), and for other heart diseases (0, 1, ≥2).

Health characteristic need factors included: body mass index (BMI, <18.5, 18.5-< 25, 25-<30, ≥30 kg/m²), dyspnea, urinary incontinence (continent, incontinent, or not rated [catheter, ostomy or no urine output]), fall history in the month before the index SNF admission (yes, no, or unable to determine), pressure ulcers (stage ≥1), presence of pain (yes/no), and level of depression severity based on the Patient Health Questionnaire (PHQ)-9 or PHQ-9 Observational Version. Cut-points of 5, 10, 15, and 20 defined none, mild, moderate, moderately severe, and severe depression, respectively.²¹

Functional status need factors included an assessment of patient’s physical limitations and cognitive impairment. Physical limitation was assessed based on the MDS activities of daily living score²² and categorized as either normal or minimal limitations (0–2), moderate limitations (3–4), or physical dependency (5–6). We included variables for six individual activities such as eating, dressing, and bathing. Each activity was rated on a scale from independence, supervised, limited assistance, extensive assistance, or total dependency (including only did activity once or twice/didn’t do activity in the past 7 days). Cognitive impairment was measured based on the self-reported Brief Interview for Mental Status or a staff-reported Cognitive Performance Scale (normal or minimal impairment, moderate impairment, severe impairment).^23–25

We included 5-star overall quality rating, 5-star nursing staffing rating, certified bed size (<100, 100–299, ≥300 beds), chain membership (Yes/No), profit orientation (for profit/non-profit), geographic location (urban or rural county), hours per resident per day of total nursing care (quartiles), and hours per resident per day of licensed nursing care (quartiles).

We determined 30-day all-cause mortality based on MBSF. Using MedPAR data, 30-day re-hospitalization was defined as any new non-elective acute care hospitalization, excluding admissions for rehabilitation, within 30 days after the index SNF admission.

We randomly split the sample into a development cohort (two-thirds) and a validation cohort (one-third). The validation cohort was used to test the discrimination ability of the prediction models.

We examined continuous factors including patient age, Charlson index, and length of index hospital stay for possible nonlinear effects through a generalized additive model (GAM) in relation to 30-day mortality and re-hospitalizations separately. Based on the GAM results, we modeled age and log-transformed Charlson index as a linear function for mortality and as a non-parametric function using a smoothing spline for the re-hospitalization outcome. The length of the index hospital stay was categorized as 3–4, 5–6, 7–9, 10–13, ≥14 days based on their prediction curves for both outcomes.

We set two criteria for variable selection; i) p-value <0.1 and ii) area under the receiver operating characteristic curve (AUC, i.e., C-statistic) increase >0.01.²⁶ To assess predictive factors for the study outcomes, we sequentially investigated and added covariates to the logistic regression models (Models 1–6), beginning with predisposing factors (Model 1), next adding comorbidity need factor variables (Model 2), need factor variables for health service use (Model 3), need factor variables for health characteristics (Model 4), need factor variables for functional status (Model 5), and then facility-level variables (Model 6). We examined the clustering effects of SNFs as random effects of SNFs in the final model for each outcome and evaluated whether AUCs were improved.

Calibration was assessed using the Hosmer and Lemeshow (H-L) test for all models without the random effects of SNFs (p >0.05 indicating good calibration). Because using the H-L test in a hierarchical logistic regression model may not be valid due to correlation within clusters, we visually assessed the extent of agreement between observed and predicted values. Predicted values of each outcome were sorted in ascending order and equally divided into 20 groups. We considered an indication of lack-of-fit when the differences between observed and predicted values were not random.

For validity, we calculated the AUCs for each model in the validation cohort. An AUC of 0.5 indicates no discrimination and 1 represents perfect discrimination. We consider AUCs of 0.60–0.69 to indicate limited discrimination, AUCs of 0.70 – 0.79 to indicate modest discrimination, and AUCs ≥0.80 to indicate good discrimination.²⁷ In addition, to compare the contribution to prediction power for each factor in the final model, we calculated the percentage of decrease in prediction power if the corresponding factor was absent in the development cohort using the following formula: $\frac{AUC \left(Final model\right)-AUC \left(Model without corresponding factor\right)}{AUC \left(Final model\right)-0.5}*100\%$. The percent of decrease is 100% when a model only includes the intercept.

We conducted a series of sensitivity analyses to examine the robustness of the predictive models. We estimated the AUCs in models with all predictors which were associated with outcomes and had positive contributions to the AUCs. We estimated the AUC in residents without missing data. We estimated the AUCs in residents aged ≥65 years, stratified by sex. We estimated the AUC for 30-day re-hospitalization among residents who survived 30 days after SNF admission. We used R 3.4.0 with mgcv package for GAM models and gamm4 package for a generalized additive mixed model (www.r-project.org).

Results

From 11,529 SNFs, 51,783 residents were in the development and 25,887 residents were in the validation cohort (see Appendix Figure 1 for patient selection details). In the development cohort, the median age of residents was 84 years, 39.6% were men, 83.4% were non-Hispanic whites, and 19.1% were dually enrolled in Medicaid. The median Charlson index was 4 and the median length of index hospital stay was 6 days. Forty-three percent had dyspnea and 25.8% had depression. Eighty percent had moderate physical limitations or dependency and 35.9% had moderate or severe cognitive impairment. Approximately one-half lived in a SNF with a 4- or 5-star rating for registered nurse staffing. Both patient-level and facility-level characteristics of the validation cohort were similar to those in the development cohort (Tables 1–2, Appendix Table 1).

Table 1.

Predisposing factors, comorbidities, and health service use of the development and validation cohorts

Characteristics, % or median (25th - 75th percentiles)	Development cohort (n = 51,783)	Validation cohort (n = 25,887)
Predisposing factors
Age, years	84 (77 – 89)	84 (78 – 89)
Men	39.6	40.2
Race/Ethnicity
Non-Hispanic white	83.4	83.3
Non-Hispanic black	9.9	10.1
Hispanic	2.9	2.8
Non-Hispanic other	1.4	1.5
Unknown	2.4	2.3
Marital status
Married	30.5	31.1
Widowed	50.8	50.0
Separated/divorced	8.8	9.1
Never married	8.0	7.9
Unknown	1.9	1.9
Medicare and Medicaid dual enrollment	19.1	19.1
Need factors – comorbidities
Charlson index	4 (3 – 5)	4 (3 – 5)
Type of heart failure
Systolic	39.5	39.7
Diastolic	35.3	35.3
Unspecified	25.2	24.9
Hypertension	79.5	79.5
Coronary artery disease	62.6	62.8
Atrial fibrillation	56.6	57.2
Cerebrovascular disease	7.2	6.9
Peripheral vascular disease	11.1	11.1
Hyperlipidemia	43.2	43.4
Diabetes	39.7	39.9
COPD or asthma	34.0	33.8
Anemia	30.4	30.4
Chronic kidney disease	34.8	35.5
Dementia	18.8	18.6
Depression	24.6	24.6
Pneumonia	14.7	15.1
Urinary tract infection	12.1	12.6
Need factors – health service use
Length of index hospital stay, days	6 (4 – 9)	6 (4 – 9)
Intensive care unit stay during index hospitalization	38.0	38.0
No. of hospitalizations for heart failure within 6 months before index hospitalization
0	82.1	82.3
1	13.2	13.2
≥ 2	4.6	4.5
No. of hospitalizations for other heart diseases within 6 months before index hospitalization
0	92.0	92.1
1	7.1	7.0
≥ 2	0.9	0.9
No. of any hospitalizations within 6 months before index hospitalization
0	56.9	57.6
1	25.8	25.6
≥ 2	17.3	16.8

Table 2.

Health characteristics and functional status of the development and validation cohorts

Characteristics, %	Development cohort (n = 51,783)	Validation cohort (n = 25,887)
Need factors – health characteristics
Body mass index, kg/m2
< 18.5	6.0	5.9
18.5-< 25	35.9	35.7
25-< 30	25.8	26.2
≥ 30	29.1	28.9
Unknown	3.3	3.3
Dyspnea	43.4	43.2
Urinary incontinence
Continent	69.8	70.1
Incontinent	23.5	23.0
Not rated	6.6	7.0
Fall (previous 30 days before skill nursing facility admission)
Yes	20.6	21.0
No	71.2	70.9
Unable to determine	8.2	8.2
Pressure ulcers	16.7	16.6
Pain	57.4	57.6
Depression severity (PHQ-9 Score)
None	74.2	73.7
Mild	17.3	17.8
Moderate	4.9	4.7
Moderately severe	1.4	1.4
Severe	0.4	0.4
Unknown	1.9	2.0
Need factors – functional status
Physical limitations
Normal or minimal limitation	20.0	19.7
Moderate limitation	61.9	62.0
Physical dependency	18.1	18.3
Cognitive function
Normal or minimal impairment	64.1	64.3
Moderate impairment	24.3	24.1
Severe impairment	11.6	11.5
Eating
Independent	39.4	39.9
Supervision	39.8	39.3
Limited assistance	12.2	12.2
Extensive assistance	6.9	6.8
Total dependency	1.7	1.7
Dressing
Independent	1.9	1.9
Supervision	4.3	4.2
Limited assistance	23.7	23.3
Extensive assistance	66.0	66.5
Total dependency	4.1	4.1
Bathing
Independent	1.1	1.0
Supervision	2.6	2.5
Limited assistance	7.9	7.8
Extensive assistance	69.1	69.6
Total dependency	19.2	19.1
Toileting
Independent	1.9	2.0
Supervision	4.4	4.5
Limited assistance	21.6	21.4
Extensive assistance	67.0	67.2
Total dependency	5.0	5.0
Bed mobility
Independent	4.4	4.6
Supervision	5.3	5.4
Limited assistance	22.3	22.0
Extensive assistance	65.4	65.4
Total dependency	2.6	2.6
Locomotion on unit
Independent	3.6	4.0
Supervision	8.0	8.0
Limited assistance	24.5	23.6
Extensive assistance	46.1	46.4
Total dependency	17.8	18.0

After the SNF admission, 6.8% of residents died and 24.2% were re-hospitalized within 30 days in the development cohort. Of those who died, 56.1% were re-hospitalized and 13.6% were discharged to home or self-care before their death. Of those who were re-hospitalized, 18.5% were discharged to home or self-care before their readmission. In the validation cohort, 6.9% died and 24.3% were re-hospitalized within 30 days after the SNF admission similar to those in the development cohort.

The AUC for 30-day mortality in the development cohort in Model 1 with all predisposing factors except Medicare and Medicaid dual eligibility was 0.594. Adding the log-transformed Charlson index increased the AUC to 0.609; no other comorbidities were selected in Model 2. Adding the length of index hospital stay increased the AUC to 0.619; no other health service use factors were selected in Model 3. Adding BMI, dyspnea, urinary incontinence, having a pressure ulcer, and depression severity further increased the AUC to 0.691 in Model 4. Adding physical limitations and cognitive impairment increased the AUC to 0.707 in Model 5. None of the facility-level factors was selected in Model 6. There was no evidence of a lack of fit in all models based on the H-L tests. The AUCs for each model in the validation cohort were similar to those in the development cohort (Table 3).

Table 3.

Area under receiver operating characteristic curves (AUCs) for 30-day all-cause mortality

Models	Predictors	Development cohort		Validation cohort
		AUCs (95% CI)	H-L test*	AUCs (95% CI)
1	Age (linear), gender, race/ethnicity, marital status	0.594 (0.585–0.604)	0.19	0.580 (0.567–0.594)
2	Model 1 + Charlson index (log-transformed,linear)	0.609 (0.600–0.618)	0.91	0.600 (0.586–0.613)
3	Model 2 + length of index hospital stay (5 categories)	0.619 (0.610–0.629)	0.68	0.613 (0.600–0.627)
4	Model 3 + BMI, dyspnea, urinary incontinence,pressure ulcer, depression severity	0.691 (0.682–0.700)	0.16	0.695 (0.682–0.707)
5 (Final model)	Model 4+ physical limitations, cognitive impairment	0.707 (0.699–0.716)	0.49	0.708 (0.695–0.720)
6	No facility-level factor met the selection criteria	NA	NA	NA

^*H-L test: Hosmer and Lemeshow test (p >0.05 indicating good calibration)

Although adding the random effects of SNFs in Model 5 increased the AUC to 0.750 in the development cohort, the AUC was only 0.709 in the validation cohort. The observed and predicted values were visually close to each other in Model 5, regardless of taking into account the random effects of SNFs or not in the validation cohort (Appendix Figure 2). Because adding the random effects of SNFs in Model 5 did not markedly improve the AUC and calibration in the validation cohort, we selected Model 5 without random effects of SNFs as the final model for 30-day mortality; the β-coefficients for each predictor are listed in Appendix Table 2. Among the selected predictors, developing symptoms of dyspnea, having physical limitations, depression severity, BMI (decreased mortality risk in residents with BMI ≥25) and Charlson index contributed most to prediction power (Figure 1).

Figure 1.

Decrease in prediction power for 30-day mortality and 30-day re-hospitalization

Top: 30-day mortality model

Bottom: 30-day re-hospitalization model

The AUC for 30-day re-hospitalization in the development cohort in Model 1 was 0.553. Adding the log-transformed Charlson index increased the AUC to 0.583; no other comorbidities were selected in Model 2. Adding the length of index hospital stay and the number of hospitalizations during the past 6 months increased the AUC to 0.607 in Model 3. Depression severity in Model 4 further increased the AUC to 0.621. Adding physical limitations slightly increased the AUC to 0.626 in Model 5. None of the facility-level factors was selected in Model 6. The p values for the H-L tests were >0.05 for all models. The AUCs for each model in the validation cohort were similar to those in the development cohort.

Adding the random effects of SNFs in Model 5 increased the AUC to 0.667 in the development but not in the validation cohort (AUC = 0.625). The visual agreement between observed and predicted values in Model 5 with random effects of SNFs was not as good as that in Model 5 without the inclusion of random effects of SNFs (Appendix Figure 2). Model 5 without inclusion of the random effects of SNFs was selected as the final model for 30-day re-hospitalization; the β-coefficients for each predictor except age and Charlson Index are listed in Appendix Table 3A. Among the selected predictors, depression severity, number of hospitalizations in past 6 months, length of the index hospitalization, Charlson index, and physical limitations contributed most to prediction power (Figure 1).

All sensitivity analyses yielded similar AUCs for 30-day mortality and for 30-day re-hospitalization as those in the original analyses (see Appendix Table 4).

Discussion

Routinely collected administrative patient-level data can be used to predict 30-day mortality with modest predictive accuracy and 30-day re-hospitalization with more limited predictive accuracy. Some patient-level factors including dyspnea, physical limitations, depression severity, number of prior hospitalizations, and length of index hospital stay were important predictors for either 30-day mortality or 30-day re-hospitalization, or both. No facility-level factors predicted these outcomes.

We found that 6.8% of patients died and 24.2% were re-hospitalized within 30 days. These estimates are slightly lower than previous reports.⁴ Most of HF prediction models/risk scores, including CMS’s,²⁸ predict 30-day mortality after hospital admission (thus including deaths during hospitalization).⁹ A limited number of recent studies that have attempted to predict 30-day post-discharge mortality in residents with HF demonstrated moderate to good discrimination (C-statistics: 0.66–0.86), but none used national data.^29–32 Studies with good discrimination (C-statistics >0.80) included multiple key laboratory and vital sign variables^{29, 32} which were not in our data. At best, our AUC is 0.7, which demonstrates modest discrimination. It is likely because the key discriminative parameters are not included in the administrative data used (e.g., changes in functional status, orthostatic hypotension, etc.).

Charlson index, length of index hospital stay, depression, and cognitive function are important predictors for 30-day post-discharge mortality in patients with HF,^{29, 30, 32} and BMI (protective effect of increased BMI) is consistently predicts mortality.⁹ Our study affirmed these findings in a SNF population. Although some comorbidities, such as diabetes, have been included in various prediction models for mortality in patients with HF,¹⁰ none of the specific clinical diagnoses contributed to meaningful additional prediction ability when the model included the Charlson index. We found that dyspnea and physical limitations were important predictors for 30-day mortality. Activities of daily living have been shown to be an important prognostic risk factor in elderly patients with HF.³³ Both dyspnea and physical limitations are likely proxies for the severity of HF, advanced age, or both. However, dyspnea has multiple causes, many of which are not heart related (e.g., anxiety).

Models used to predict readmission after HF hospitalization generally demonstrate poorer discrimination than those developed to predict mortality;^{7, 9} our study is no exception. The CMS risk adjustment model based on Medicare claims data for 30-day readmission in patients with HF had a C-statistic of 0.60;³⁴ adding clinical data did not improve model discrimination.³⁵ Other models have been developed using administrative data, electronic health records, or patient-reported information to predict 30-day readmission in patients with HF. Most, including several widely-accepted HF mortality risk scores, demonstrated poor or limited discrimination in predicting hospital readmission (C-statistics: 0.54–0.65).^{29–31, 36, 37} Huynh et al. developed a model with a C-statistic of 0.80 for 30-day readmission³², but the C-statistic dropped to 0.73 when validated on a large patient cohort.³⁸ Mahajan et al. recently developed a model with a C-statistic of 0.84 for readmission in patients with HF, but this study was carried out at a single Veterans health care system (n=1,210 men) and included vital signs and laboratory measures in the model.³⁹

Re-hospitalization among patients discharged from the hospital after HF has been suggested to be the product of a much broader set of forces involving clinical and non-clinical factors (e.g., socioeconomic status, behavioral, and mental health),^{29, 30, 32} although results from the Tele-HF study suggested that non-clinical factors are not dominant factors in predicting readmission risk for patients with HF.³⁷ In our study, depression severity contributed most to the prediction ability for 30-day re-hospitalization, followed by the number of hospitalizations in the past 6 months and length of the patient’s index hospital stay. Although physical limitations were a predictor for re-hospitalization in our study, cognitive function was not. In a small study of 156 older patients from 58 SNFs with an acute HF exacerbation, the authors found that re-hospitalized patients were less likely to have dementia.⁴⁰

Current health and reimbursement policies have drawn attention to 30-day mortality and re-hospitalization for patients hospitalized with HF in the United States.⁴¹ This is a clinically meaningful period that can be strongly influenced by hospital care and early transition to the outpatient setting. Of more than 1 million hospital discharges for HF each year in the United States,¹ approximately one-quarter are discharged into SNFs.^{3, 4} SNFs have emerged as an integral component of care for Medicare beneficiaries with HF.¹² We previously reported that SNF residents with HF are old and suffer from significant physical limitations and cognitive impairment and a high degree of comorbidity.¹⁸ Therefore, predictive models of 30-day mortality and re-hospitalization in this highly vulnerable population could potentially serve as a powerful tool for clinicians and patients to make informed decisions and appropriate care planning and coordination.

Despite this, all-cause re-hospitalization is far more difficult to predict and to prevent than cause-specific re-hospitalizations, given the diversity of possible hospitalization diagnoses.⁴² The discrimination differences between the mortality and re-hospitalization models in our study reaffirm the known challenges in predicting patient risk of re-hospitalization. Furthermore, the differences in predictors between the 30-day mortality and re-hospitalization models suggest that mortality and re-hospitalization may reflect different domains of care.

To our knowledge, this is the first study to develop and validate risk prediction models for patients with HF discharged to SNFs using national data sources. Both prediction models may provide estimates of risk that can assist clinicians in counseling patients and their families and guide a potentially shared clinical decision making process. Furthermore, some modifiable risk factors such as depression and dyspnea have been identified as important predictors, which could be potential targets for intervention or management among this vulnerable population during this high-risk period.

Despite these study strengths, there are several limitations that must be kept in mind in interpreting our study results. First, our models lacked key clinical data such as ejection fraction findings and laboratory values that were not included in administrative data. Further, we did not have access to information regarding falls, delirium, and drug side effects during the SNF stay. Adding clinical data to administrative data significantly improved the prediction of mortality among patients hospitalized with HF, although it did not improve the re-hospitalization model.³⁵ Second, the MDS 3.0 did not include information on the use of do not resuscitate orders. Third, the re-hospitalization model did not take into account the competing risk due to death. However, the discrimination capacity for re-hospitalization did not change when we restricted our analyses to those who survived 30 days after their index SNF admission. Finally, although we took into account some SNF-level factors, we did not have information available about HF-specific transitional care and HF disease management approaches. That the AUCs markedly increased after taking into account random effects of SNFs in the development cohort suggests this possibility. Because 18% of SNFs contributed a single patient with HF, these facilities in the validation cohort were not presented in the development cohort, which probably explain that the predictive power in the validation cohort was not improved with random effects of SNFs.

Conclusions

Older patients with HF discharged to SNFs are a diverse population.⁴³ Preventing death and hospital readmission among these patients is challenging. Our model for mortality had modest ability to identify those at risk for dying within 30 days. Although readmission within 30 days of a prior hospitalization for HF has emerged as a major focus of quality improvement and payment reform, predicting re-hospitalization is a more daunting task than predicting mortality. Since the model for re-hospitalization had limited discrimination, there is a need for further understanding of the factors that influence re-hospitalization risk.

Table 4.

Area under receiver operating characteristic curves (AUCs) for 30-day all-cause re-hospitalization

Models	Predictors	Development cohort		Validation cohort
		AUCs (95% CI)	H-L test*	AUCs (95% CI)
1	Age (spline), gender, race/ethnicity, marital status, Medicare and Medicaid dual eligibility	0.553 (0.548–0.559)	0.21	0.555 (0.547–0.563)
2	Model 1 + Charlson index (log-transformed,spline)	0.583 (0.577–0.588)	0.08	0.585 (0.577–0.594)
3	Model 2 + length of index hospital stay (5 categories), No. of any hospitalizations within 6 months before index hospitalization	0.607 (0.601–0.612)	0.56	0.608 (0.600–0.616)
4	Model 3 + depression severity	0.621 (0.615–0.626)	0.41	0.619 (0.611–0.627)
5 (Final model)	Model 4 + physical limitations	0.626 (0.620–0.632)	0.15	0.625 (0.617–0.633)
6	No facility-level factor met the selection criteria	NA	NA	NA

^*H-L test: Hosmer and Lemeshow test (p >0.05 indicating good calibration)

Appendix

Appendix Figure 1.

Selection of Medicare patients discharged to a skilled nursing facility after an acute hospitalization for heart failure

Appendix Figure 2.

Calibration curves for observed versus predicted probability of 30-day mortality in the validation cohort

Left: Model 5 for 30-day mortality without random effects of SNFs (GLM = generalized linear model)

Right: Model 5 for 30-day mortality with random effects of SNFs (GLMM = generalized linear mixed model)

Appendix Figure 3.

Calibration curves for observed versus predicted probability of 30-day re-hospitalization in the validation cohort

Left: Model 5 for 30-day re-hospitalization without random effects of SNFs (GAM = generalized additive model)

Right: Model 5 for 30-day re-hospitalization with random effects of SNFs (GAMM = generalized additive mixed model)

Appendix Table 1.

Facility-level characteristics of the development and validation cohorts

Characteristics, %	Development cohort (n = 51,783)	Validation cohort (n = 25,887)
5-star overall quality rating
1	10.9	11.0
2	17.7	17.6
3	19.9	20.0
4	29.0	28.7
5	21.3	21.3
Unknown	1.3	1.4
5-star registered nurse staffing rating
1	7.9	8.6
2	15.6	15.5
3	26.4	25.7
4	24.9	24.5
5	21.8	22.2
Unknown	3.4	3.5
Certified bed size
< 100	34.1	34.1
100 – 299	54.7	55.0
≥ 300	11.2	11.0
Chain membership
Yes	56.8	56.4
No	43.2	43.6
Profit orientation
For profit	67.8	67.8
Non-profit	32.2	32.2
Geographic location
Urban area	84.2	83.9
Rural area	15.8	16.1
Hours per resident per day of total nursing care
0 - < 3.66 (quartile 1)	23.2	23.4
3.66 - < 4.08 (quartile 2)	23.4	23.1
4.08 -< 4.67 (quartile 3)	23.8	23.4
≥ 4.67 (quartile 4)	23.5	23.7
Unknown	6.1	6.3
Hours per resident per day of licensed nursing care
0 - < 1.39 (quartile 1)	23.4	23.4
1.39 - < 1.62 (quartile 2)	23.3	23.5
1.62 -< 1.94 (quartile 3)	23.5	23.0
≥ 1.94 (quartile 4)	23.6	23.7
Unknown	6.1	6.3

Appendix Table 2.

Final predictors for 30-day all-cause mortality

Parameter/Predictor	β-coefficient	SE	p value
Intercept	−5.841	0.238	<.0001
Age	0.019	0.003	<.0001
Gender (reference: women)
Men	0.323	0.040	<.0001
Race/Ethnicity (reference: Non-Hispanic white)
Non-Hispanic black	−0.435	0.074	<.0001
Hispanic	−0.452	0.124	0.0003
Non-Hispanic other	−0.178	0.152	0.24
Unknown	−0.017	0.118	0.89
Marital status (reference: Married)
Widowed	−0.062	0.044	0.16
Separated/divorced	−0.213	0.078	0.007
Never married	−0.103	0.078	0.19
Unknown	0.145	0.131	0.27
Charlson index (log-transformed)	0.321	0.035	<.0001
Length of index hospital stay, days (reference: 3–4)
5–6	0.063	0.051	0.215
7–9	0.210	0.051	<.0001
10–13	0.274	0.060	<.0001
≥ 14	0.275	0.066	<.0001
Body mass index, kg/m2 (reference: 18.5 - < 25)
< 18.5	0.209	0.069	0.0026
25 - < 30	−0.205	0.045	<.0001
≥ 30	−0.505	0.051	<.0001
Unknown	0.100	0.091	0.27
Dyspnea	0.555	0.036	<.0001
Urinary incontinence (reference: continent)
Incontinent	0.258	0.043	<.0001
Not rated	0.406	0.061	<.0001
Pressure ulcers	0.291	0.043	<.0001
Depression severity (reference: none)
Mild	0.211	0.046	<.0001
Moderate	0.486	0.071	<.0001
Moderately severe	0.692	0.117	<.0001
Severe	0.781	0.211	0.0002
Unknown	1.172	0.091	<.0001
Physical limitations (reference: normal or minimal limitation)
Moderate limitation	0.510	0.062	<.0001
Physical dependency	0.931	0.069	<.0001
Cognitive function (reference: normal or minimal impairment)
Moderate impairment	0.226	0.043	<.0001
Severe impairment	0.482	0.053	<.0001

Appendix Table 3A.

Final predictors for 30-day all-cause re-hospitalization where age and Charlson Index were modeled as non-parametric function

Parameter/Predictor	β-coefficient	SE	p value
Intercept	−1.601	0.120	<.0001
Age*	NA	NA	NA
Gender (reference: women)
Men	0.071	0.023	0.0021
Race/Ethnicity (reference: Non-Hispanic white)
Non-Hispanic black	0.032	0.036	0.38
Hispanic	−0.014	0.062	0.82
Other	0.213	0.085	0.012
Unknown	−0.023	0.069	0.74
Marital status (reference: Married)
Widowed	−0.020	0.026	0.44
Separated/divorced	−0.081	0.041	0.05
Never married	−0.081	0.043	0.059
Unknown	0.015	0.078	0.84
Medicare and Medicaid dual enrollee	0.021	0.029	0.47
Charlson index (log-transformed)*	NA	NA	NA
Length of index hospital stay, days (reference: 3–4)
5–6	0.127	0.029	<.0001
7–9	0.227	0.030	<.0001
10–13	0.363	0.036	<.0001
≥ 14	0.517	0.038	<.0001
No. of any hospitalizations within 6 months before index hospitalization (ref: 0)
1	0.173	0.026	<.0001
≥ 2	0.471	0.029	<.0001
Depression severity (reference: none)
Mild	0.020	0.028	0.47
Moderate	0.026	0.049	0.59
Moderately severe	0.036	0.089	0.68
Severe	0.217	0.159	0.17
Unknown	1.673	0.069	<.0001
Physical limitations (reference: normal or minimal limitation)
Moderate limitation	0.149	0.028	<.0001
Physical dependency	0.422	0.034	<.0001

^*Because age and log-transformed Charlson Index were treated as a non-parametric function using a smoothing spline, parameter estimates were not applicable.

Appendix Table 3B.

Final predictors for 30-day all-cause re-hospitalization where age and Charlson Index were modeled as categorical variables

Parameter/Predictor	β-coefficient	SE	p value
Intercept	−1.773	0.065	<.0001
Age, years (reference: <65)
65–69	0.038	0.064	0.55
70–74	0.020	0.060	0.74
75–79	0.023	0.058	0.69
80–84	0.003	0.056	0.96
85–89	−0.063	0.057	0.27
90–94	−0.111	0.060	0.062
≥95	−0.285	0.072	<.0001
Gender (reference: women)
Men	0.082	0.023	0.0004
Race/Ethnicity (reference: Non-Hispanic white)
Non-Hispanic black	0.036	0.036	0.32
Hispanic	−0.011	0.062	0.86
Other	0.213	0.085	0.012
Unknown	−0.023	0.069	0.73
Marital status (reference: Married)
Widowed	−0.021	0.027	0.43
Separated/divorced	−0.081	0.041	0.051
Never married	−0.090	0.043	0.036
Unknown	0.014	0.078	0.86
Medicare and Medicaid dual enrollee	0.023	0.029	0.44
Charlson index (reference: 1–4)
5–7	0.211	0.024	<.0001
>7	0.270	0.0426	<.0001
Length of index hospital stay, days (reference: 3–4)
5–6	0.132	0.029	<.0001
7–9	0.238	0.030	<.0001
10–13	0.376	0.036	<.0001
≥ 14	0.533	0.038	<.0001
No. of any hospitalizations within 6 months before index hospitalization (ref: 0)
1	0.194	0.025	<.0001
≥ 2	0.506	0.029	<.0001
Depression severity (reference: none)
Mild	0.023	0.028	0.41
Moderate	0.027	0.049	0.58
Moderately severe	0.036	0.089	0.68
Severe	0.216	0.159	0.18
Unknown	1.669	0.069	<.0001
Physical limitations (reference: normal or minimal limitation)
Moderate limitation	0.151	0.028	<.0001
Physical dependency	0.424	0.034	<.0001

Appendix Table 4.

Area under receiver operating characteristic curves (AUCs) for 30-day mortality and 30-day re-hospitalization in sensitivity analyses

Sensitivity analyses	Predictors	Mortality: AUCs (95% CI)		Re-hospitalization: AUCs (95% CI)
		Development Cohort	Validation Cohort	Development Cohort	Validation Cohort
Original study population	All predictors* associated with the outcomes and increased AUC	0.729 (0.721–0.738)	0.727 (0.715–0.739)	0.650 (0.644–0.655)	0.642 (0.634–0.650)
Completed cases without missing data	Predictors in final models	0.702 (0.692–0.711)	0.702 (0.689–0.715)	0.623 (0.617–0.629)	0.621 (0.612–0.629)
Patients aged ≥ 65 years	Predictors in final models	0.704 (0.695–0.712)	0.707 (0.694–0.719)	0.625 (0.620–0.631)	0.623 (0.615–0.632)
Men	Predictors in final models except gender	0.711 (0.699–0.724)	0.686 (0.667–0.704)	0.619 (0.611–0.628)	0.620 (0.607–0.632)
Women	Predictors in final models except gender	0.693 (0.681–0.706)	0.713 (0.696–0.730)	0.629 (0.622–0.637)	0.627 (0.616–0.638)
Patients survived 30 days after SNF admission	Predictors in final models	NA	NA	0.627 (0.621–0.634)	0.623 (0.615–0.632)

^*For 30-day all-cause mortality, in addition to the predictors in the final model, the newly added predictors included type of heart failure, hypertension, coronary artery disease, atrial fibrillation, hyperlipidemia, diabetes, anemia, chronic kidney disease, depression, pneumonia, number of hospitalizations for heart failure and number of any hospitalizations in the 6 months before the index hospitalization, fall history in the month before the index SNF admission, pain, individual activity for eating, dressing, using the toilet, bathing, getting in and out of bed, and locomotion on unit, separately, as well as the facility-level factors (5-star nursing staffing rating, certified bed size, and geographic location); The Hosmer and Lemeshow test p = 0.56 in the development cohort.

For 30-day all-cause re-hospitalization, in addition to the predictors in the final model, the newly added predictors included type of heart failure, hypertension, coronary artery disease, atrial fibrillation, hyperlipidemia, diabetes, COPD/asthma, anemia, chronic kidney disease, dementia, depression, pneumonia, number of hospitalizations for heart failure and for other heart disease in the 6 months before the index hospitalization, separately, use of intensive unit care during the index hospitalization, BMI, dyspnea, urinary incontinence, fall history in the month before the index SNF admission, pressure ulcer, individual activity for eating, dressing, using the toilet, bathing, getting in and out of bed, and locomotion on unit, separately, as well as the facility-level factors (5-star nursing staffing rating, certified bed size, chain membership, profit orientation, geographic location, hours per resident per day of total nursing care, and hours per resident per day of licensed nursing care); The Hosmer and Lemeshow test p = 0.11 in the development cohort