Do Non-Clinical Factors Improve Prediction of Readmission Risk? Results from the Tele-HF Study

Abstract

Background

Existing readmission risk models have poor discrimination and it is unknown whether they would be markedly improved by the inclusion of patient-reported information.

Objectives

We sought to determine if a model that included self-reported socioeconomic, health status, and psychosocial characteristics obtained from patients recently discharged from hospitalizations for heart failure substantially improved 30-day readmission risk prediction compared with a model that incorporated only clinical and demographic factors.

Methods

As part of the Telemonitoring to Improve Heart Failure Outcomes (Tele-HF) trial, we conducted medical record abstraction and telephone interviews in a sample of 1,004 patients recently hospitalized for heart failure to obtain clinical, functional, and psychosocial information within 2 weeks of discharge. Candidate risk factors included 110 variables divided into 2 groups: demographic and clinical variables generally available from the medical record; and socioeconomic, health status, adherence, and psychosocial variables from patient interview.

Results

The 30-day readmission rate was 17.1%. Using the 3-level risk score derived from the restricted medical record variables, patients with a score of 0 (no risk factors) had a readmission rate of 10.9% (95% CI 8.2%, 14.2%) and patients with a score of 2 (all risk factors) had a readmission rate of 32.1% (95% CI 22.4%, 43.2%), C-statistic 0.62. Using the 5-level risk score derived from all variables, patients with a score of 0 (no risk factors) had a readmission rate of 9.6% (95% CI 6.1%, 14.2%) and patients with a score of 4 (all risk factors) had a readmission rate of 55.0% (95% CI 31.5%, 76.9%), C-statistic 0.65.

Conclusions

Self-reported socioeconomic, health status, adherence, and psychosocial variables are not dominant factors in predicting readmission risk for patients with heart failure. Patient-reported information improved model discrimination and extended the predicted ranges of readmission rates, but the model performance remained poor.

INTRODUCTION

Preventing readmissions after heart failure hospitalizations is a national priority, but the risk of readmission is difficult to predict. In a survey of readmission risk scores, Kansagara et al. reported that most risk models have poor discrimination and predictive ability (1). In the model that is publicly reported by the Centers for Medicare & Medicaid Services and is part of the Hospital Readmissions Reduction Program, the discrimination of a model using administrative claims as well as the medical record model used for validation was less than 0.70 (2).

A potential explanation for the poor discrimination of these models is that patient factors beyond clinical and basic demographic characteristics, which are the principal components of these models, may play an important role in readmission risk. Most models have not included information from patient interviews that could characterize information about their socioeconomic, health status, adherence, and psychosocial characteristics. Whether this information would markedly improve the model performance is not known.

Accordingly, we sought to determine whether a readmission risk model that incorporated information obtained from the patient, including clinical, socioeconomic, health status, and psychosocial characteristics, could improve risk prediction compared with a model that incorporated only clinical and demographic factors. We supplemented information available from the medical record at the time of discharge with information from a patient interview and used it to develop a risk score that could be compared with a model built only from data available at discharge.

METHODS

Study Sample

Data for these analyses were derived from our published trial to assess the effect of telemonitoring on patients with heart failure (Telemonitoring to Improve Heart Failure Outcomes – Tele-HF) (3,4). The primary outcome of Tele-HF was readmission or death from any cause within 180 days, and there were no differences between study arms (telemonitoring vs. usual care) in rates of readmission, death, or the combined endpoint of either death or readmission. Because there was also no difference in readmission rate at 30 days, the current analysis included patients combined from both arms of Tele-HF. Tele-HF enrolled 1,653 patients who had been hospitalized for heart failure in the previous 30 days (“index admission”) at 33 study sites in the United States. Exclusion criteria included age <18 years; long-term nursing home residence; being a prison inmate; inability to participate in the study protocol, including irreversible medical conditions likely to affect 6-month survival, inability to stand on a scale, severe cognitive impairment (Folstein score <20) (5) and no access to telephone service; chronic hemodialysis; severe aortic or mitral valve heart disease; enrollment in another disease management study; and, since the primary outcome includes all-cause hospitalization, plans for an inpatient cardiac procedure. In addition to the Tele-HF exclusions, we excluded patients who were not interviewed between 3 and 30 days post-discharge (N = 574) (all patients except one had a baseline interview; this exclusion was due to interviews outside of the window established for this study) or who were readmitted between discharge and interview (N = 36) and, to ensure that patients could be scored, those who were missing >15 of the 110 candidate variables (N=39). The final sample for this study included 1,004 participants. The Human Investigation Committee at the Yale University School of Medicine approved the study.

Data Collection

We obtained baseline data through medical record review and patient interview. Site coordinators abstracted medical records for clinical information. The Coordinating Center at Yale University sought to conduct interviews with patients to obtain clinical, functional, and psychosocial information. The median time from discharge to the interview was 12 days (Interquartile Range 6–19).

Outcomes

The outcome was hospital readmission for any cause within 30 days after the interview. Readmission was assessed through medical record review, patient interviews conducted at 3 and 6 months post-enrollment, and direct contact with area hospitals, including the index admission hospital. We used these 3 sources to identify discrepancies concerning readmission status or date and resolved them by contacting the relevant hospitals.

We ascertained mortality status for enrolled patients after the conclusion of the 180-day follow-up period. For patients who did not have a record of death in the medical chart, and who were not able to be contacted directly for the follow-up survey after 180 days, we determined vital status by searching the Social Security Death Index, contacting other residents of the patients’ households, and searching online obituaries for patients of the same name and date of birth in the same city. We used date of death to censor patients in time-to-readmission analyses; all surviving patients were censored at 30 days.

Variables

Tele-HF included collection of several hundred clinical, demographic, treatment, and psychosocial data elements for each patient, as described previously (3). For scales that were comprised of multiple items (e.g., the Kansas City Cardiomyopathy Questionnaire [KCCQ](6)), we included individual items rather than summary scores and, as a secondary analysis, replicated the analyses using the full scales. We further excluded variables that were missing in >20% of the study sample (income category, number of previous admissions for heart failure, brain natriuretic peptide, physician follow-up scheduled, and 8 of the KCCQ items). The remaining set of candidate risk factors included 110 variables (Appendix). These risk factors were divided into 2 groups: demographic and clinical variables that are generally available from the medical record; and socioeconomic, health status, and psychosocial variables that are not generally available but might improve the predictive power of a risk model and be collectable, if clinically important.

Statistical Analysis

We summarized the characteristics of the included and excluded patients and compared the 2 groups using χ² tests of independence. We next sought to develop the most parsimonious model of the highest predictive value from the available patient variables; first, using only the demographic and clinical variables, and then using all available patient variables. For each of the 110 included variables, we estimated a single Cox proportional hazards model with time-to-readmission as the outcome, censored for death. We used these results to collapse multi-category responses into fewer categories, where appropriate, based on frequency of the response, the face validity of a combination, and similarity of the association with the outcome.

Then, to reduce the resulting set of variables to a subset that was most predictive of 30-day readmission, we used a random forest (RF) algorithm (7,8). In an RF algorithm, an iterative process involving random selection is used to assign weights to each variable considered. First, a random bootstrap sample is drawn from the full set of observations; then, random subsets of 10 variables are drawn and compared on some metric. In our case, we used a Cox proportional hazards model with time-to-readmission as the outcome and assigned a score to each variable according to the standardized effect size. At each step, the best-scored variable moved on to the next stage, until a final set of weights was calculated for each variable. This is repeated over random bootstrap samples and the weight for each variable is averaged over all random samples to produce an importance weight (IW). The advantages of the RF algorithm include: the IW assigned a variable by RF is not sensitive to correlation or interaction with other variables; many more variables can be scored using RF than can be assessed using multivariable or stepwise regression techniques; the RF algorithm incorporates split-sample validation at each step; and the random sample and random variable selection provide a robust treatment of missing data.

To assign an IW to each variable, we used a version of RF known as a random survival forest (RSF) algorithm (9). For each random subset of variables, a Cox proportional hazards model is estimated, with time-to-readmission as the outcome and censoring for death at 30 days. Weight is then determined by the absolute magnitude of the coefficient from the regression model (9). For our analysis, we selected 10 random variables at each step and used multiple imputation with 20 imputations to account for missing data in each Cox regression; this was repeated for 2,000 randomly selected samples and variables.

The result of the RSF analysis was a relative IW for each variable under consideration, reported as a percentage of the IW of the most important variable. For further consideration, we retained variables with relative IWs of at least 20%, indicating that they were at least one-fifth as important in predicting 30-day readmission as the most important variable. Because importance weights are calculated independently of each other, we further reduced this set of variables by applying forward stepwise selection to a Cox regression model, including at each step the variable with the greatest t-value (most significant) as long as the level of significance was <5%. For stepwise selection, we restricted to only those patients with no missing data for the retained variables. Stepwise regression is known to produce over-narrow confidence intervals and artificially small P-values,(10,11) and applying stepwise regression applied after RSF may furthermore bias the P-values up or down; for this reason, while we used the P-values to identify predictors, we caution against using them to make inferences. Using the final set of variables, we estimated a Cox regression model using multiple imputation to account for missing values (12). Finally, to construct the score, we assigned, for each of the final risk factors, a number of points consistent with the magnitude of the corresponding hazard ratio from this final model. We replicated the entire RSF analysis, stepwise selection, final model, and score construction using (a) only demographic and clinical risk factors available on hospital discharge and (b) the set of demographic and clinical risk factors plus all additional psychometric and socioeconomic measures. As a secondary analysis, we replicated the analysis using psychometric scales rather than individual items.

We evaluated each of the 2 risk scores by reporting the observed 30-day readmission rate for each value of the risk score and calculating the C-statistic for each. To assess whether the probability of readmission increased with increasing risk score, we performed a test for trend. We compared any nested models by calculating the integrated discrimination improvement (13). Finally, because these data were for patients enrolled in a trial, we compared the final scores by intervention group using a rank-sum test. We also created a score for those individuals who had interviews within 2 weeks.

We performed the analyses in R version 3.0.1 (9,14) and Stata version 13.1 (StataCorp 2014, College Station, TX).

RESULTS

Description of the Study Sample

There were 1,653 patients enrolled in the study, of which 574 were not interviewed between 3 and 30 days post-discharge; 36 were readmitted before their interview; and 39 were missing more than 15 of the 110 variables, leaving 1,004 patients in the sample. The included patients were similar to the excluded (Table 1) with the exception of age, for which included patients were older (P < 0.001); rate of readmission within 30 days of baseline interview did not differ (P = 0.09). The mean age of the group was 62 years, with 341 (34.0%) younger than age 55. The sample had just over 41% women and almost 40% were African-American (Table 1). The majority of the patients had New York Heart Association Class II or III heart failure on admission and about 70% had a left ventricular ejection fraction <40%. Comorbidities were common; three quarters of the subjects had hypertension and nearly half had diabetes mellitus. The 30-day mortality rate from the time of the interview was 4.9% and the 30-day readmission rate from the time of the interview was 17.1%.

Table 1.

Patient (included and excluded) characteristics.

	Excluded	Included	P-value
N	649 (39.3)	1004 (60.7)
Age			0.0002
≤55	262 (40.4)	341 (34.0)
56–64	143 (22.0)	210 (20.9)
65–74	99 (15.3)	242 (24.1)
≥75	144 (22.2)	211 (21.0)
Missing	1 (0.2)	0 (0.0)
Sex			0.4510
Male	368 (56.7)	589 (58.7)
Female	280 (43.1)	415 (41.3)
Missing	1 (0.2)	0 (0.0)
Race			0.0889
White	309 (47.6)	507 (50.5)
African American	250 (38.5)	393 (39.1)
Other	90 (13.9)	104 (10.4)
Hispanic			0.2583
No	634 (97.7)	973 (96.9)
Yes	14 (2.2)	31 (3.1)
Missing	1 (0.2)	0 (0.0)
New York Heart Association class			0.2030
1	42 (6.5)	58 (5.8)
2–3	556 (85.7)	889 (88.5)
4	50 (7.7)	57 (5.7)
Missing	1 (0.2)	0 (0.0)
Left ventricular ejection fraction			0.4799
Normal	183 (28.2)	289 (28.8)
20–39	442 (68.1)	671 (66.8)
<20	6 (0.9)	16 (1.6)
Missing	18 (2.8)	28 (2.8)
Glomerular filtration rate			0.2504
0–30	93 (14.3)	119 (11.9)
31–60	250 (38.5)	416 (41.4)
>60	297 (45.8)	451 (44.9)
Missing	9 (1.4)	18 (1.8)
Chronic renal failure			0.9567
No	482 (74.3)	748 (74.5)
Yes	166 (25.6)	256 (25.5)
Missing	1 (0.2)	0 (0.0)
Chronic pulmonary edema			0.8740
No	511 (78.7)	795 (79.2)
Yes	137 (21.1)	209 (20.8)
Missing	1 (0.2)	0 (0.0)
Diabetes			0.0784
No	363 (55.9)	518 (51.6)
Yes	285 (43.9)	486 (48.4)
Missing	1 (0.2)	0 (0.0)
Hypertension			0.0700
No	165 (25.4)	217 (21.6)
Yes	483 (74.4)	787 (78.4)
Missing	1 (0.2)	0 (0.0)
Coronary artery disease; myocardial infarction; ischemic cardiomyopathy			0.1431
No	335 (51.6)	482 (48.0)
Yes	313 (48.2)	522 (52.0)
Missing	1 (0.2)	0 (0.0)
Angiotensin-converting enzyme inhibitor or angiotensin receptor-blocker			0.2290
No	226 (34.8)	321 (32.0)
Yes	423 (65.2)	683 (68.0)
Beta-blocker			0.7601
No	137 (21.1)	206 (20.5)
Yes	511 (78.7)	798 (79.5)
Missing	1 (0.2)	0 (0.0)
Loop diuretic			0.6073
No	137 (21.1)	223 (22.2)
Yes	511 (78.7)	781 (77.8)
Missing	1 (0.2)	0 (0.0)
Digoxin			0.2261
No	476 (73.3)	764 (76.1)
Yes	172 (26.5)	240 (23.9)
Missing	1 (0.2)	0 (0.0)
Aldosterone antagonist			0.8295
No	433 (66.7)	676 (67.3)
Yes	215 (33.1)	328 (32.7)
Missing	1 (0.2)	0 (0.0)
Readmission within 30 days of baseline interview			0.0939
No	557 (85.8)	832 (82.9)
Yes	91 (14.0)	172 (17.1)
Missing	1 (0.2)	0 (0.0)
Death within 30 days of baseline interview			0.6275
No	639 (98.5)	987 (98.3)
Yes	9 (1.4)	17 (1.7)
Missing	1 (0.2)	0 (0.0)

Risk Score

Of the final set of 110 variables considered for potential inclusion in the risk model, 27 were classified as demographic or clinical (Appendix). After applying the RSF algorithm to the set of 27 variables for the 1,004 patients, 5 variables had a relative importance of at least 20% (Table 2). Forward stepwise Cox regression using these variables found that only 1 obtained at the index admission (blood urea nitrogen (BUN) level) had an independent effect on readmission with a significance level of P <0.05 (Table 3). Repeating this process using all 110 variables identified 7 with a relative importance of at least 20%; forward stepwise Cox regression retained 3 of these: BUN; reported swelling (KCCQ-3); and reported shortness of breath (Tables 3 and 4).

Table 2.

Results of random survival forest analysis using 27 demographic and clinical variables.

Variable	Importance	Relative Importance
Blood urea nitrogen	0.006	1.000
Glomerular filtration rate	0.002	0.396
Female	0.002	0.386
Waist/Hip ratio	0.002	0.288
Medical history - ischemic cardiomyopathy	0.001	0.206
Medical history - permanent pacemaker	0.001	0.181
Medical history - illicit drug use	0.001	0.137
Medical history - coronary artery disease	0.001	0.130
Medical history - prior myocardial infarction	0.001	0.117
Medical history - cerebrovascular disease/prior stroke	0.001	0.112
Hispanic	0.001	0.084
Systolic blood pressure	0.000	0.056
Smoking status	0.000	0.048
Left ventricular ejection fraction	0.000	0.029
New York Heart Association	0.000	0.026
Race	0.000	0.001
Pulmonary	0.000	−0.061
Jugular venous distention	0.000	−0.062
Waist (inches)	−0.001	−0.102
Symptoms - tiredness/fatigue	−0.001	−0.123
Medical history - hypertension	−0.001	−0.162
Pitting edema	−0.001	−0.191
Medical history - diabetes	−0.001	−0.231
Medical history - chronic pulmonary disease	−0.002	−0.264
Body mass index	−0.002	−0.354
Rapid Estimate of Adult Literacy in Medicine (REALM)	−0.002	−0.408
Age	−0.003	−0.535

Table 3.

Results of random survival forest analysis using all (110) variables.

Variable	Importance	Relative Importance
Blood urea nitrogen	0.0054	1.0000
KCCQ: Past two weeks swelling bother you	0.0026	0.4758
Glomerular filtration rate	0.0014	0.2539
SF-12	0.0013	0.2479
KCCQ: Bathing yourself	0.0013	0.2426
KCCQ: Past 2 weeks has shortness of breath bothered you	0.0011	0.2047
KCCQ: Limited intimate relationships	0.0011	0.2031
Systolic blood pressure	0.0010	0.1819
PSS: Confident to handle problems	0.0009	0.1666
KCCQ: Past 2 weeks did you have swelling	0.0009	0.1657
KCCQ: Physical	0.0008	0.1524
KCCQ: In last 2 weeks have symptoms changed	0.0007	0.1312
Economic burden	0.0007	0.1292
KCCQ: Symptoms limited your work	0.0007	0.1245
KCCQ: Last 2 weeks felt discouraged	0.0007	0.1230
Medical history - chronic pulmonary disease	0.0006	0.1167
KCCQ: Limited visiting family	0.0006	0.1143
KCCQ: Dressing yourself	0.0006	0.1104
PHQ9: Speaking very slowly	0.0006	0.1034
KCCQ: Past two weeks has fatigue bothered you	0.0005	0.0901
Morisky: Forgotten your medications	0.0005	0.0877
KCCQ: Doing yard work	0.0005	0.0872
Ware: Doctors ignore what I tell them	0.0004	0.0800
Medical history - coronary artery disease	0.0004	0.0693
ESSI: Someone you are close to	0.0004	0.0668
Medical history - permanent pacemaker	0.0003	0.0640
ESSI: Give you love and affection	0.0003	0.0627
PHQ9: Feeling bad about yourself	0.0003	0.0608
Medical history - illicit drug use	0.0003	0.0566
KCCQ: Limited your hobbies	0.0003	0.0560
PHQ9: Feeling down	0.0003	0.0502
KCCQ: Climbing a flight of stairs	0.0003	0.0477
Pitting edema	0.0003	0.0467
Live alone	0.0002	0.0460
KCCQ: Hurrying or jogging	0.0002	0.0456
KCCQ: Past two weeks has fatigue limited you	0.0002	0.0450
Jugular venous distention	0.0002	0.0412
PHQ9: Trouble concentrating	0.0002	0.0390
Education	0.0002	0.0354
Ware: Get medical care whenever I need	0.0002	0.0332
REALM: REALM-R card to the patient - constipation	0.0002	0.0314
Financially how are you	0.0002	0.0289
REALM: REALM-R card to the patient - osteoporosis	0.0001	0.0278
Ware: Doctor treats me friendly	0.0001	0.0270
REALM: REALM-R card to the patient - fatigue	0.0001	0.0252
Ware: Medical care is perfect	0.0001	0.0251
ESSI: Give you emotional support	0.0001	0.0245
Ware: Doctor spends plenty of time with me	0.0001	0.0190
Hispanic	0.0001	0.0176
Ware: Access to specialists	0.0001	0.0166
Medical history - ischemic cardiomyopathy	0.0001	0.0165
KCCQ: Past two weeks has shortness of breath limited you	0.0001	0.0153
PHQ9: Trouble sleeping	0.0001	0.0151
Ware: Have to wait too long for emergency	0.0001	0.0144
Waist (inches)	0.0001	0.0141
Waist/Hip ratio	0.0001	0.0120
Ware: Careful to check everything	0.0001	0.0096
Smoking status	0.0000	0.0075
Medical history - prior myocardial infarction	0.0000	0.0042
ESSI: Give you good advice	0.0000	0.0036
KCCQ: Understand how to keep from getting worse	0.0000	0.0016
Currently have a doctor for your health care	0.0000	0.0016
Morisky: If feeling worse then stop medication	0.0000	0.0003
PSS: Unable to control important things in life	0.0000	0.0002
Race	0.0000	−0.0027
KCCQ: Past two weeks had to sleep sitting up	0.0000	−0.0039
Difficult to get care	0.0000	−0.0040
PHQ9: Better off dead	0.0000	−0.0048
Medical history - cerebrovascular disease/prior stroke	0.0000	−0.0062
KCCQ: Walking one block	0.0000	−0.0067
ESSI: Help with daily chores	0.0000	−0.0077
New York Heart Association	0.0000	−0.0078
Morisky: Stop taking your medication	0.0000	−0.0080
PHQ9: Feeling tired	0.0000	−0.0086
Ware: Doctor explains well	0.0000	−0.0092
REALM: REALM-R card to the patient - colitis	−0.0001	−0.0096
REALM: REALM-R card to the patient - anemia	−0.0001	−0.0100
Ware: Dissatisfied with medical care	−0.0001	−0.0113
PSS: Difficulties are piling up	−0.0001	−0.0134
Avoided health care due to cost	−0.0001	−0.0139
REALM	−0.0001	−0.0152
REALM: REALM-R card to the patient - allergic	−0.0001	−0.0153
PHQ9: Poor appetite	−0.0001	−0.0165
Pulmonary	−0.0001	−0.0171
Ware: Doubts about doctor treating me	−0.0001	−0.0191
Low cognition	−0.0001	−0.0196
PSS: Things are going your way	−0.0001	−0.0204
Symptoms - tiredness/fatigue	−0.0001	−0.0208
PHQ9: Little interest in doing things	−0.0001	−0.0223
KCCQ: Rest of life with heart failure	−0.0001	−0.0225
Body mass index	−0.0001	−0.0226
Ware: Doctor acts too businesslike	−0.0001	−0.0238
Ware: Hurry too much when treating me	−0.0001	−0.0245
Female	−0.0001	−0.0250
REALM: REALM-R card to the patient - jaundice	−0.0001	−0.0267
Ware: Doctor’s office has everything needed	−0.0001	−0.0271
KCCQ: Whom to call if getting worse	−0.0002	−0.0290
Medical history - hypertension	−0.0002	−0.0296
KCCQ: Limited enjoyment of life	−0.0002	−0.0313
Morisky: Careless about taking medication	−0.0002	−0.0314
REALM: REALM-R card to the patient - directed	−0.0002	−0.0419
Work status	−0.0002	−0.0424
Medical history - diabetes	−0.0003	−0.0485
Ware: Hard to get an appointment	−0.0003	−0.0496
Left ventricular ejection fraction	−0.0003	−0.0514
Ware: Receive care without financial worry	−0.0003	−0.0554
Have health insurance	−0.0003	−0.0561
ESSI: Someone available when need to talk	−0.0004	−0.0791
Ware: Pay more than I can afford	−0.0006	−0.1032
Age	−0.0009	−0.1620

ESSI, ENRICHD Social Support Instrument; KCCQ, Kansas City Cardiomyopathy Questionnaire; PHQ, Patient Health Questionnaire; PSS, Perceived Stress Scale; REALM, Rapid Estimate of Adult Literacy in Medicine; SF12, 12-Item Short Form Health Survey

Table 4.

Results of forward stepwise selection Cox model (n=1004).

Variable	Clinical and demographic only			All risk factors
	HR (95% CI)	P-value	Wald P	HR (95% CI)	P-value	Wald P
Blood urea nitrogen			<0.001			<0.001
≤20	referent			referent
21–50	0.72 (0.36, 1.08)	<0.001		0.71 (0.35, 1.07)	<0.001
>50	1.27 (0.78, 1.75)	<0.001		1.23 (0.75, 1.72)	<0.001
KCCQ: Past 2 weeks did you have swelling						0.015
No/Doesn’t apply				ref
Yes				−0.41 (−0.74, −0.08)	0.015
KCCQ: Past 2 weeks has shortness of breath bothered you						0.014
No/Doesn’t apply				ref
Yes				−0.43 (−0.77, −0.09)	0.014

CI, confidence interval; HR, hazard ratio; KCCQ, Kansas City Cardiomyopathy Questionnaire

Considering the magnitude of the hazard ratios in Table 4, we assigned each patient 1 point for each of: reporting that his/her health was an economic burden; reporting swelling in the last 2 weeks; reporting health status of “Poor”; systolic blood pressure ≤90; and BUN >20. We assigned an additional point to patients with BUN >50. To reflect real-world applications in which not all information might be available, we assigned 0 to a risk factor that was either negative or missing.

Table 5 illustrates the 30-day readmission rate for each category of risk score derived from each set of risk factors. Using the risk score derived from the restricted set of commonly available variables, patients with a score of 0 (no risk factors) had a readmission rate of 10.9% (95% CI 8.2%, 14.2%) while patients with a score of 2 had a readmission rate of 32.1% (95% CI 22.4%, 43.2%) with a C-statistic of 0.62. In comparison, using the risk score derived from all variables, patients with a score of 0 (no risk factors) had a readmission rate of 9.6% (95% CI 6.1%, 14.2%) and patients with a score of 4 had a readmission rate of 55.0% (95% CI 31.5%, 76.9%) with a C-statistic of 0.65. The test for trend found a positive trend for both risk scores (P <0.001).

Table 5.

Risk scores.

Score	N	30-day readmission % (95% CI)
Derived from 27 demographic and clinical variables
0	457	10.9 (8.2, 14.2)
1	463	20.5 (16.9, 24.5)
2	84	32.1 (22.4, 43.2)
C-statistic, 0.6240
Derived from 110 demographic, clinical, psychometric, and socioeconomic variables.
0	228	9.6 (6.1, 14.2)
1	385	13.2 (10.0, 17.0)
2	275	21.8 (17.1, 27.2)
3	96	29.2 (20.3, 39.3)
4	20	55.0 (31.5, 76.9)
C-statistic, 0.6496

CI, confidence interval

DISCUSSION

Our principal finding is that even with the inclusion of a number of patient-centered variables obtained shortly after admission, there was only minor improvement in the discrimination of a risk model to predict 30-day readmissions after a post-discharge interview following a heart failure hospitalization. Although our potential predictors were much more extensive than those used in previous studies and our outcomes were validated, we were unable to develop models with high discrimination. Our results reveal that the limitations in predicting readmission do not stem from not having information about the patient’s symptoms, health status, psychosocial characteristics, access to health care, or economic status.

The discrimination in our model that was based on all available variables is much lower than what has been achieved in mortality models. For example, our discrimination was much lower than that reported by Lee and colleagues, who developed a mortality model for patients hospitalized with heart failure that was derived from basic demographic and detailed clinical variables and had a C-statistic of 0.80 for 30-day mortality (15). Prior reviews of readmission models for patients hospitalized with heart failure have reported discrimination performance that is comparable to that of the 2 models presented in this study (1,16). Our previous efforts with basic demographic and detailed clinical data yielded similar results even though we employed different methods (2,17).

The explanation for the poor discrimination of the models is not known. Unmeasured factors related to health system quality may play a prominent role, as many system interventions have been shown to reduce readmission and gaps in the quality of transitional care are common (18–20). Discharged patients may have an acquired, transient syndrome of generalized risk, which is not represented well by the characteristics that we included and may depend more on the allostatic stress experienced during hospitalization (21). The pronounced variation in the causes of readmission suggests that the severity of the condition leading to the hospitalization is not the only factor that influences the risk of readmission (22). The inherent propensity of a system to admit patients, which is not incorporated into these models, might be the dominant influence (23) and bed supply, which may be a mediator of the propensity for admission, may also play a role (24). The inclusion criteria of first the randomized controlled trial and then of this study likely resulted in a more homogeneous sample than most that are typically used to develop risk models; if so, then there would be less variation in risk factors and outcomes, and consequently reduced discrimination. However, it is worth noting that the study population included almost 40% African Americans and 10% other non-white groups and had substantial diversity with respect to socioeconomic status. We enrolled patients from 33 sites across the country and our event rates are similar to nationally reported rates. Lastly, despite the breadth of variables that we included, other unidentified patient-level variables, such as the quality of the discharge summary, may be responsible for the readmission risk as has been recently reported (25,26).

Our model does not perform as well as a single-center model, developed by Amarasingham and colleagues, which used data from an electronic health record but not from patient interviews (27). The model included non-clinical factors such as number of home address changes and missed clinic visits. Their discrimination, at 0.72, was higher with these variables but still not as high as mortality models. Their model may be conveying information about utilization behavior and barriers to health care access - or may also carry quality of care information.

In our model based on all the variables, self-reported lower extremity swelling and health status were identified as important predictors. The reason why these variables were more important than heart failure severity is not clear. Since all patients had decompensated heart failure requiring hospitalization, it may be that severity of heart failure was not discriminating risk among the sample. Interestingly, age, race and other prominent socioeconomic variables were not sufficiently predictive to be included in the models, including reported medication adherence. Our study sample included a diverse range of patients and had good representation by age, race, and socioeconomic status. Our findings suggest that these socioeconomic variables do not carry much weight in predicting readmission when viewed with other detailed information about the patient.

A strength of our study is the novel application of an RSF algorithm to avoid the known bias in stepwise and other automated variable selection processes, and validation of the final subset of variables selected from the large number of variables collected. This method, robust to the presence of nonlinear effects and complex interactions, has been found to produce highly predictive models (28,29).

Nevertheless, our findings should be interpreted in the context of several potential limitations. The sample was derived from a clinical trial population consisting of individuals who agreed to participate and who may be more adherent than patients who were not enrolled in a clinical trial. Although the score should be validated in different populations, the factors are consistent with what has been reported in other groups. The interview was conducted either during the hospitalization or shortly thereafter and the reference time was different across the sample. Nevertheless, we assessed the outcomes from the time of the patient-reported information and so the patients were stratified at the point that they were providing feedback about themselves. There is also the limitation of sample size. In this cohort, a risk factor that is present in 30% of the patients would only be detectable in bivariate analysis with 80% power if it elevated the risk of readmission by 7.5%. However, though smaller effects may be clinically meaningful, it is arguable that very small effects would not be of interest in a prognostic tool.

In conclusion, we failed to demonstrate that expanded demographic and patient-reported information could markedly improve the performance of readmission risk models. The patient factors related to health and demographics seem inadequate and there is a need for further understanding of the factors that dominantly influence readmission risk. These factors may include health system quality of care, hospitalization stress, and propensity to admit.

PERSPECTIVES: CORE CLINICAL COMPETENCIES AND TRANSLATIONAL IMPLICATIONS.

Competency in Medical Knowledge

Readmission risk for patients is difficult to predict from demographic, clinical and patient self-reported information.

Competency in Patient Care

After hospitalization, clinicians should be aware that the risk of readmission is high and it is difficult to stratify this risk further with conventionally available data.

Translational Outlook 1

In practice, there is a need to recognize that risk-stratification of patients for their risk of readmission is challenging. Even the lowest risk patients have a substantial risk.

Translational Outlook 2

Clinicians should recognize the limitations of the current readmission models and appreciate that there are likely unmeasured factors that may be providing a strong influence on patient recovery.

ABBREVIATIONS

BUN

blood urea nitrogen

CI

confidence interval

IW

importance weight

KCCQ

Kansas City Cardiomyopathy Questionnaire

RSF

random survival forest

Tele-HF

Telemonitoring to Improve Heart Failure Outcomes

RF

random forest

Appendix. Potential risk factors included in the analysis

	Frequency	30-day Readmission
N	1004 (100.0)	172 (17.1)
Age
≤55	341 (34.0)	55 (16.1)
56–64	210 (20.9)	38 (18.1)
65–74	242 (24.1)	43 (17.8)
≥75	211 (21.0)	36 (17.1)
Female
No	589 (58.7)	114 (19.4)
Yes	415 (41.3)	58 (14.0)
Race
White	507 (50.5)	97 (19.1)
African American	393 (39.1)	63 (16.0)
Other	104 (10.4)	12 (11.5)
Hispanic
No	973 (96.9)	165 (17.0)
Yes	31 (3.1)	7 (22.6)
Medical history - cerebrovascular disease/prior stroke
No	906 (90.2)	154 (17.0)
Yes	98 (9.8)	18 (18.4)
Medical history – chronic pulmonary disease
No	795 (79.2)	146 (18.4)
Yes	209 (20.8)	26 (12.4)
Coronary artery disease
No	574 (57.2)	86 (15.0)
Yes	430 (42.8)	86 (20.0)
Diabetes
No	518 (51.6)	82 (15.8)
Yes	486 (48.4)	90 (18.5)
Hypertension
No	217 (21.6)	37 (17.1)
Yes	787 (78.4)	135 (17.2)
Illicit drug use
No	963 (95.9)	163 (16.9)
Yes	41 (4.1)	9 (22.0)
Ischemic cardiomyopathy
No	764 (76.1)	121 (15.8)
Yes	240 (23.9)	51 (21.3)
Permanent pacemaker
No	867 (86.4)	144 (16.6)
Yes	137 (13.6)	28 (20.4)
Prior myocardial infarction
No	742 (73.9)	120 (16.2)
Yes	262 (26.1)	52 (19.8)
KCCQ physical
≤35	113 (11.3)	25 (22.1)
36–80	363 (36.2)	59 (16.3)
81–100	399 (39.7)	58 (14.5)
Missing	129 (12.8)	30 (23.3)
Low cognition
No	921 (91.7)	157 (17.0)
Yes	83 (8.3)	15 (18.1)
Education
<High school	251 (25.0)	55 (21.9)
High school+	743 (74.0)	115 (15.5)
Missing	10 (1.0)	2 (20.0)
SF-1
1–3	461 (45.9)	60 (13.0)
4	379 (37.7)	73 (19.3)
5	149 (14.8)	38 (25.5)
Missing	15 (1.5)	1 (6.7)
Economic burden
Little/No burden	348 (34.7)	45 (12.9)
Some burden	638 (63.5)	124 (19.4)
Missing	18 (1.8)	3 (16.7)
Avoided health care due to cost
Yes	170 (16.9)	27 (15.9)
No	813 (81.0)	140 (17.2)
Missing	21 (2.1)	5 (23.8)
Have health insurance
Yes	762 (75.9)	133 (17.5)
No	231 (23.0)	37 (16.0)
Missing	11 (1.1)	2 (18.2)
Difficult to get care
Sometimes	211 (21.0)	36 (17.1)
Never	782 (77.9)	135 (17.3)
Missing	11 (1.1)	1 (9.1)
Work status
Do not work	822 (81.9)	148 (18.0)
Work for pay	165 (16.4)	19 (11.5)
Missing	17 (1.7)	5 (29.4)
Live alone
No	627 (62.5)	107 (17.1)
Yes	349 (34.8)	62 (17.8)
Missing	28 (2.8)	3 (10.7)
Financially how are you
Comfortable, have more than enough	233 (23.2)	42 (18.0)
Have enough to make ends meet	450 (44.8)	75 (16.7)
Do not have enough to make ends	281 (28.0)	48 (17.1)
Missing	40 (4.0)	7 (17.5)
Symptoms - tiredness/fatigue
No	916 (91.2)	158 (17.2)
Yes	88 (8.8)	14 (15.9)
Pitting edema
Yes	438 (43.6)	84 (19.2)
No	555 (55.3)	87 (15.7)
Unsure	11 (1.1)	1 (9.1)
Waist/Hip ratio
≤.9	167 (16.6)	26 (15.6)
.9–1	698 (69.5)	117 (16.8)
>1	97 (9.7)	22 (22.7)
Missing	42 (4.2)	7 (16.7)
Waist (inches)
<32	132 (13.1)	22 (16.7)
33–46	631 (62.8)	104 (16.5)
>46	241 (24.0)	46 (19.1)
Pulmonary
Bases/Above	248 (24.7)	45 (18.1)
Clear	756 (75.3)	127 (16.8)
Jugular venous distention
Present	134 (13.3)	25 (18.7)
Not present	782 (77.9)	132 (16.9)
Unsure	88 (8.8)	15 (17.0)
Glomerular filtration rate
No-30	119 (11.9)	29 (24.4)
31–60	416 (41.4)	79 (19.0)
>60	451 (44.9)	61 (13.5)
Missing	18 (1.8)	3 (16.7)
REALM
≤6	371 (37.0)	61 (16.4)
>6	633 (63.0)	111 (17.5)
New York Heart Association
1	58 (5.8)	4 (6.9)
2–3	889 (88.5)	156 (17.5)
4	57 (5.7)	12 (21.1)
Smoking status
Never	783 (78.0)	135 (17.2)
Smoked	207 (20.6)	34 (16.4)
Missing	14 (1.4)	3 (21.4)
Left ventricular ejection fraction
Normal	289 (28.8)	47 (16.3)
20–39	671 (66.8)	113 (16.8)
<20	16 (1.6)	6 (37.5)
Missing	28 (2.8)	6 (21.4)
Systolic blood pressure
≤90	66 (6.6)	18 (27.3)
91–105	208 (20.7)	42 (20.2)
106–120	266 (26.5)	36 (13.5)
121–135	213 (21.2)	40 (18.8)
>135	251 (25.0)	36 (14.3)
Body mass index
≤24.9	230 (22.9)	44 (19.1)
≤29.9	272 (27.1)	44 (16.2)
≥30	500 (49.8)	84 (16.8)
Missing	2 (0.2)	0 (0.0)
Blood urea nitrogen
≤20	422 (42.0)	45 (10.7)
21–50	463 (46.1)	95 (20.5)
>50	84 (8.4)	27 (32.1)
Missing	35 (3.5)	5 (14.3)
Currently have a doctor for your health care
Yes	833 (83.0)	142 (17.0)
No	149 (14.8)	26 (17.4)
Missing	22 (2.2)	4 (18.2)
KCCQ: Dressing yourself
NA	19 (1.9)	5 (26.3)
Yes-3	161 (16.0)	39 (24.2)
4–5	812 (80.9)	127 (15.6)
Missing	12 (1.2)	1 (8.3)
KCCQ: Bathing yourself
NA	36 (3.6)	7 (19.4)
Yes-3	168 (16.7)	41 (24.4)
4–5	789 (78.6)	123 (15.6)
Missing	11 (1.1)	1 (9.1)
KCCQ: Walking one block
NA	168 (16.7)	38 (22.6)
Yes-3	327 (32.6)	61 (18.7)
4–5	493 (49.1)	70 (14.2)
Missing	16 (1.6)	3 (18.8)
KCCQ: Doing yard work
NA	341 (34.0)	76 (22.3)
Yes-3	319 (31.8)	52 (16.3)
4–5	334 (33.3)	42 (12.6)
Missing	10 (1.0)	2 (20.0)
KCCQ: Climbing a flight of stairs
NA	377 (37.5)	81 (21.5)
Yes-3	337 (33.6)	58 (17.2)
4–5	283 (28.2)	33 (11.7)
Missing	7 (0.7)	0 (0.0)
KCCQ: Hurrying or jogging
NA	819 (81.6)	153 (18.7)
Yes-3	131 (13.0)	17 (13.0)
4–5	40 (4.0)	2 (5.0)
Missing	14 (1.4)	0 (0.0)
KCCQ: In last 2 weeks have symptoms changed
No	263 (26.2)	59 (22.4)
Yes	730 (72.7)	113 (15.5)
Missing	11 (1.1)	0 (0.0)
KCCQ: Past two weeks did you have swelling
No	377 (37.5)	81 (21.5)
Yes	606 (60.4)	86 (14.2)
Missing	21 (2.1)	5 (23.8)
KCCQ: Past two weeks swelling bother you
No	227 (22.6)	57 (25.1)
Yes	756 (75.3)	111 (14.7)
Missing	21 (2.1)	4 (19.0)
KCCQ: Past two weeks has fatigue limited you
No	557 (55.5)	109 (19.6)
Yes	430 (42.8)	61 (14.2)
Missing	17 (1.7)	2 (11.8)
KCCQ: Past two weeks has fatigue bothered you
No	471 (46.9)	96 (20.4)
Yes	507 (50.5)	71 (14.0)
Missing	26 (2.6)	5 (19.2)
KCCQ: Past two weeks has shortness of breath limited you
No	466 (46.4)	92 (19.7)
Yes	523 (52.1)	77 (14.7)
Missing	15 (1.5)	3 (20.0)
KCCQ: Past two weeks has shortness of breath bothered you
No	445 (44.3)	93 (20.9)
Yes	538 (53.6)	72 (13.4)
Missing	21 (2.1)	7 (33.3)
KCCQ: Past two weeks had to sleep sitting up
No	392 (39.0)	76 (19.4)
Yes	595 (59.3)	89 (15.0)
Missing	17 (1.7)	7 (41.2)
KCCQ: Whom to call if getting worse
No	153 (15.2)	22 (14.4)
Yes	824 (82.1)	142 (17.2)
Missing	27 (2.7)	8 (29.6)
KCCQ: Understand how to keep from getting worse
No	162 (16.1)	31 (19.1)
Yes	824 (82.1)	138 (16.7)
Missing	18 (1.8)	3 (16.7)
KCCQ: Limited enjoyment of life
No	543 (54.1)	103 (19.0)
Yes	444 (44.2)	67 (15.1)
Missing	17 (1.7)	2 (11.8)
KCCQ: Rest of life with heart failure
No	632 (62.9)	122 (19.3)
Yes	345 (34.4)	43 (12.5)
Missing	27 (2.7)	7 (25.9)
KCCQ: Last 2 weeks felt discouraged
No	466 (46.4)	89 (19.1)
Yes	521 (51.9)	77 (14.8)
Missing	17 (1.7)	6 (35.3)
KCCQ: Limited intimate relationships
NA	513 (51.1)	100 (19.5)
1–3	259 (25.8)	42 (16.2)
4–5	222 (22.1)	28 (12.6)
Missing	10 (1.0)	2 (20.0)
KCCQ: Limited visiting family
NA	210 (20.9)	42 (20.0)
1–3	281 (28.0)	52 (18.5)
4–5	501 (49.9)	75 (15.0)
Missing	12 (1.2)	3 (25.0)
KCCQ: Limited your work
NA	209 (20.8)	50 (23.9)
1–3	471 (46.9)	85 (18.0)
4–5	315 (31.4)	35 (11.1)
Missing	9 (0.9)	2 (22.2)
KCCQ: Limited your hobbies
NA	267 (26.6)	49 (18.4)
Yes-3	456 (45.4)	85 (18.6)
4–5	277 (27.6)	37 (13.4)
Missing	4 (0.4)	1 (25.0)
Ware: Doctor explains well
No	125 (12.5)	20 (16.0)
Yes	872 (86.9)	150 (17.2)
Missing	7 (0.7)	2 (28.6)
Ware: Doctor’s office has everything needed
No	97 (9.7)	16 (16.5)
Yes	898 (89.4)	153 (17.0)
Missing	9 (0.9)	3 (33.3)
Ware: Medical care is perfect
No	168 (16.7)	33 (19.6)
Yes	825 (82.2)	137 (16.6)
Missing	11 (1.1)	2 (18.2)
Ware: Receive care without financial worry
No	242 (24.1)	42 (17.4)
Yes	745 (74.2)	128 (17.2)
Missing	17 (1.7)	2 (11.8)
Ware: Careful to check everything
No	100 (10.0)	15 (15.0)
Yes	884 (88.0)	154 (17.4)
Missing	20 (2.0)	3 (15.0)
Ware: Pay more than I can afford
No	440 (43.8)	73 (16.6)
Yes	551 (54.9)	96 (17.4)
Missing	13 (1.3)	3 (23.1)
Ware: Access to specialists
No	133 (13.2)	14 (10.5)
Yes	854 (85.1)	156 (18.3)
Missing	17 (1.7)	2 (11.8)
Ware: Have to wait too long for emergency
No	314 (31.3)	59 (18.8)
Yes	666 (66.3)	109 (16.4)
Missing	24 (2.4)	4 (16.7)
Ware: Doctor acts too businesslike
No	139 (13.8)	25 (18.0)
Yes	853 (85.0)	144 (16.9)
Missing	12 (1.2)	3 (25.0)
Ware: Doctor treats me friendly
No	34 (3.4)	7 (20.6)
Yes	958 (95.4)	163 (17.0)
Missing	12 (1.2)	2 (16.7)
Ware: Hurry too much when treating me
No	226 (22.5)	38 (16.8)
Yes	762 (75.9)	131 (17.2)
Missing	16 (1.6)	3 (18.8)
Ware: Doctors ignore what I tell them
No	223 (22.2)	45 (20.2)
Yes	767 (76.4)	124 (16.2)
Missing	14 (1.4)	3 (21.4)
Ware: Doubts about doctor treating me
No	140 (13.9)	25 (17.9)
Yes	837 (83.4)	143 (17.1)
Missing	27 (2.7)	4 (14.8)
Ware: Doctor spends plenty of time with me
No	216 (21.5)	38 (17.6)
Yes	769 (76.6)	130 (16.9)
Missing	19 (1.9)	4 (21.1)
Ware: Hard to get an appointment
No	217 (21.6)	37 (17.1)
Yes	762 (75.9)	133 (17.5)
Missing	25 (2.5)	2 (8.0)
Ware: Dissatisfied with medical care
No	190 (18.9)	34 (17.9)
Yes	793 (79.0)	134 (16.9)
Missing	21 (2.1)	4 (19.0)
Ware: Get medical care whenever I need
No	121 (12.1)	22 (18.2)
Yes	865 (86.2)	148 (17.1)
Missing	18 (1.8)	2 (11.1)
REALM: REALM-R card to the patient - fatigue
No	229 (22.8)	41 (17.9)
Yes	775 (77.2)	131 (16.9)
REALM: REALM-R card to the patient - jaundice
No	256 (25.5)	47 (18.4)
Yes	748 (74.5)	125 (16.7)
REALM: REALM-R card to the patient - directed
No	137 (13.6)	22 (16.1)
Yes	867 (86.4)	150 (17.3)
REALM: REALM-R card to the patient - allergic
No	175 (17.4)	31 (17.7)
Yes	829 (82.6)	141 (17.0)
REALM: REALM-R card to the patient - colitis
No	382 (38.0)	61 (16.0)
Yes	622 (62.0)	111 (17.8)
REALM: REALM-R card to the patient - constipation
No	178 (17.7)	27 (15.2)
Yes	826 (82.3)	145 (17.6)
REALM: REALM-R card to the patient - anemia
No	212 (21.1)	41 (19.3)
Yes	792 (78.9)	131 (16.5)
REALM: REALM-R card to the patient - osteoporosis
No	339 (33.8)	61 (18.0)
Yes	665 (66.2)	111 (16.7)
Morisky: Forgotten your medications
No	827 (82.4)	152 (18.4)
Yes	177 (17.6)	20 (11.3)
Morisky: Careless about taking medication
No	886 (88.2)	153 (17.3)
Yes	118 (11.8)	19 (16.1)
Morisky: Stop taking your medication
No	958 (95.4)	165 (17.2)
Yes	46 (4.6)	7 (15.2)
Morisky: If feeling worse then stop medication
No	928 (92.4)	156 (16.8)
Yes	76 (7.6)	16 (21.1)
PHQ9: Better off dead
No	921 (91.7)	154 (16.7)
Yes	66 (6.6)	18 (27.3)
Missing	17 (1.7)	0 (0.0)
PHQ9: Speaking very slowly
No	753 (75.0)	117 (15.5)
Yes	240 (23.9)	55 (22.9)
Missing	11 (1.1)	0 (0.0)
PHQ9: Trouble concentrating
No	732 (72.9)	119 (16.3)
Yes	264 (26.3)	52 (19.7)
Missing	8 (0.8)	1 (12.5)
PHQ9: Feeling bad about yourself
No	693 (69.0)	114 (16.5)
Yes	299 (29.8)	57 (19.1)
Missing	12 (1.2)	1 (8.3)
PHQ9: Feeling tired
No	333 (33.2)	43 (12.9)
Yes	659 (65.6)	128 (19.4)
Missing	12 (1.2)	1 (8.3)
PHQ9: Trouble sleeping
No	484 (48.2)	69 (14.3)
Yes	510 (50.8)	102 (20.0)
Missing	10 (1.0)	1 (10.0)
PHQ9: Feeling down
No	606 (60.4)	94 (15.5)
Yes	391 (38.9)	78 (19.9)
Missing	7 (0.7)	0 (0.0)
PHQ9: Little interest in doing things
No	658 (65.5)	112 (17.0)
Yes	334 (33.3)	56 (16.8)
Missing	12 (1.2)	4 (33.3)
PHQ9: Poor appetite
No	683 (68.0)	108 (15.8)
Yes	307 (30.6)	63 (20.5)
Missing	14 (1.4)	1 (7.1)
ESSI: Someone you are close to
No	140 (13.9)	33 (23.6)
Yes	854 (85.1)	138 (16.2)
Missing	10 (1.0)	1 (10.0)
ESSI: Give you emotional support
No	153 (15.2)	33 (21.6)
Yes	839 (83.6)	137 (16.3)
Missing	12 (1.2)	2 (16.7)
ESSI: Help with daily chores
No	208 (20.7)	31 (14.9)
Yes	793 (79.0)	141 (17.8)
Missing	3 (0.3)	0 (0.0)
ESSI: Give you love and affection
No	106 (10.6)	24 (22.6)
Yes	896 (89.2)	148 (16.5)
Missing	2 (0.2)	0 (0.0)
ESSI: Give you good advice
No	168 (16.7)	29 (17.3)
Yes	832 (82.9)	142 (17.1)
Missing	4 (0.4)	1 (25.0)
ESSI: Someone available when need to talk
No	149 (14.8)	27 (18.1)
Yes	852 (84.9)	144 (16.9)
Missing	3 (0.3)	1 (33.3)
PSS: Difficulties are piling up
No	594 (59.2)	98 (16.5)
Yes	373 (37.2)	67 (18.0)
Missing	37 (3.7)	7 (18.9)
PSS: Things are going your way
No	505 (50.3)	80 (15.8)
Yes	473 (47.1)	88 (18.6)
Missing	26 (2.6)	4 (15.4)
PSS: Confident to handle problems
No	684 (68.1)	104 (15.2)
Yes	296 (29.5)	61 (20.6)
Missing	24 (2.4)	7 (29.2)
PSS: Unable to control important things in life
No	550 (54.8)	84 (15.3)
Yes	432 (43.0)	82 (19.0)
Missing	22 (2.2)	6 (27.3)

ESSI, ENRICHD Social Support Instrument; KCCQ, Kansas City Cardiomyopathy Questionnaire; REALM-R, Rapid Estimate of Adult Literacy in Medicine-Revised; PSS, Perceived Stress Scale; PHQ, Patient Health Questionnaire