A clinical score to predict mortality in patients after acute heart failure from Japanese registry

Abstract

Aims

Clinical scores that consider physical and social factors to predict long‐term observations in patients after acute heart failure are limited. This study aimed to develop and validate a prediction model for patients with acute heart failure at the time of discharge.

Methods and results

This study was retrospective analysis of the Kitakawachi Clinical Background and Outcome of Heart Failure Registry database. The registry is a prospective, multicentre cohort of patients with acute heart failure between April 2015 and August 2017. The primary outcome to be predicted was the incidence of all‐cause mortality during the 3 years of follow‐up period. The development cohort derived from April 2015 to July 2016 was used to build the prediction model, and the test cohort from August 2016 to August 2017 was used to evaluate the prediction model. The following potential predictors were selected by the least absolute shrinkage and selection operator method: age, sex, body mass index, activities of daily living at discharge, social background, comorbidities, biomarkers, and echocardiographic findings; a risk scoring system was developed using a logistic model to predict the outcome using a simple integer based on each variable's β coefficient. Out of 1253 patients registered, 1117 were included in the analysis and divided into the development (n = 679) and test (n = 438) cohorts. The outcomes were 246 (36.2%) in the development cohort and 143 (32.6%) in the test cohort. Eleven variables including physical and social factors were set into the logistic regression model, and the risk scoring system was created. The patients were divided into three groups: low risk (score 0–5), moderate risk (score 6–11), and high risk (score ≥12). The observed and predicted mortality rates were described by the Kaplan–Meier curve divided by risk group and independently increased (P < 0.001). In the test cohort, the C statistic of the prediction model was 0.778 (95% confidence interval: 0.732–0.824), and the mean predicted probabilities in the groups were low, 6.9% (95% confidence interval: 3.8–10%); moderate, 30.1% (95% confidence interval: 25.4%–34.8%); and high, 79.2% (95% confidence interval: 72.6%–85.8%). The predicted probability was well calibrated to the observed outcomes in both cohorts.

Conclusions

The Kitakawachi Clinical Background and Outcome of Heart Failure score was helpful in predicting adverse events in patients with acute heart failure over a long‐term period. We should evaluate the physical and social functions of such patients before discharge to prevent adverse outcomes.

Introduction

Heart failure (HF) is one of the most common diseases in cardiology due to its higher prevalence in the elderly and the increasing aging population. ¹ , ² Improvements in the treatment of HF are shown in many guidelines, and patients with HF have better outcomes than previous clinical setting ³ ; however, the mortality of these patients remains high. ⁴ , ⁵ Moreover, many patients with acute HF (AHF) are old and have many comorbidities, physical disorders, and social problems. ⁶ A decline in physical performance is a valuable predictor of worse outcomes in patients with HF, ⁷ , ⁸ and social problems are associated with worse outcomes in patients with AHF. ⁹

Some studies have reported risk scores in patients with chronic HF (CHF), ¹⁰ , ¹¹ and some have reported them in patients with AHF during hospitalization or short‐term observation after discharge. ¹² , ¹³ These scores were valuable and valid in Japanese patients with HF ¹⁴ , ¹⁵ ; however, physical activity and social problems were not considered in these scores, and they were evaluated in patients with CHF or short‐term observations. Therefore, clinical scores containing physical and social factors to predict long‐term observations in patients with AHF are limited. Understanding the physical and social background of these patients is important when determining their management.

The Kitakawachi Clinical Background and Outcome of Heart Failure (KICKOFF) Registry was designed as a prospective, multicentre cohort of Japanese patients with AHF. ⁶ A total of 13 hospitals in the north of Kitakawachi and Yawata, which are typical satellite communities in Osaka, Japan, participated in the study. Using this database, this study aimed to develop and validate a prediction model for patients with AHF at the time of discharge.

Methods

Study design

This study was a retrospective analysis of the database of the KICKOFF Registry, which included patients diagnosed with AHF during hospitalization between April 2015 and August 2017. The institutions participating in the study were 13 hospitals in the north of Kitakawachi (Hirakata City, Neyagawa City, and Katano City) and Yawata. Kitakawachi and Yawata are typical satellite communities in Japan, and they are located at the eastern end of the Osaka Prefecture and at the southern end of the Kyoto Prefecture, respectively. Based on the Framingham criteria, ¹⁵ HF was diagnosed when there were at least two major criteria or one major and two minor criteria. There were no exclusion criteria. The detailed study design of the KICKOFF Registry is described in the University hospital Medical Information Network (UMIN) Clinical Trials Registry (UMIN000016850). The clinical data of all patients were collected via the Internet Database System. The data were automatically verified for missing or contradictory entries and values that were not in the normal range. The data were also checked by the general office of the registry. Data from medical record reviews and interviews with patients or other family members were also recorded.

The study protocol complied with the ethical guidelines of the 1975 Declaration of Helsinki and was approved by the ethics committee of the Hirakata Kohsai Hospital (Osaka, Japan). This study has been reported according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis reporting guidelines. ¹⁶ Informed consent was obtained from all participants prior to their enrolment in the study. Direct patient identifiers were not registered to preserve patient confidentiality. The study did not alter any treatment specified in the protocol or any other method of outpatient care.

Patient data and outcomes

All patients diagnosed with AHF during hospitalization between April 2015 and August 2017 and discharged alive were included in this analysis. Of the 1253 patients registered in the KICKOFF database, 1117 patients were included for analysis and divided into the development cohort [N = 679, median [interquartile range, IQR] age: 79 [71–86] years; men: 347 (51.1%)] and the test cohort [N = 438, median [IQR] age: 79 [70–85] years; men: 224 (51.1%)] (Supporting Information, Figure S1 ). Other patient characteristics and in‐hospital data are described in Table 1 and Supporting Information, Table S1 . The outcomes were 246 (36.2%) in the development cohort and 143 (32.6%) in the test cohort.

Table 1.

Baseline clinical characteristics

	Development cohort	Test cohort
N	679	438
Sex; men (%)	347 (51)	224 (51)
Age, median (IQR), years	79.0 (71.0, 86.0)	79.0 (70.0, 85.0)
Coronary artery disease (%)	197 (29)	122 (28)
Valve disease (%)	220 (32)	120 (27)
Cardiomyopathy (%)	100 (15)	67 (15)
Hypertension (%)	463 (68)	284 (65)
Atrial fibrillation (%)	295 (43)	179 (41)
Chronic obstructive pulmonary disease (%)	102 (15)	57 (13)
Stroke (%)	81 (12)	53 (12)
Previous hospitalization for HF (%)	287 (42)	119 (27)
Current smoker (%)	81 (12)	66 (15)
Daily drinking (%)	107 (16)	87 (20)
Living alone (%)	204 (30)	122 (28)
Main drug management (not patients themselves) (%)	203 (30)	106 (24)
ADL at discharge (unable to walk independently) (%)	149 (22)	77 (18)
Active cancer (%)	21 (3.1)	25 (5.7)
BMI, median (IQR), kg/m2	21.5 (19.1, 24.2)	21.4 (19.2, 23.9)
Serum albumin, median (IQR), mg/dL	3.5 (3.1, 3.8)	3.3 (3.0, 3.7)
Haemoglobin, median (IQR), g/dL	11.6 (10.3, 13.2)	11.6 (10.3, 13.2)
eGFR, median (IQR), mL/min/1.73 m2	46 (32, 61)	48 (31, 62)
BNP, median (IQR), pg/dL	244 (116, 494)	187 (78, 460)
Left atrial dimension, median (IQR), mm	41 (36, 46)	40 (35, 45)
LVEF, median (IQR), %	53 (38, 67)	54 (39, 65)
HbA1c, median (IQR), %	6.1 (5.8, 6.6)	6.1 (5.8, 6.7)
Mortality (%)	246 (36)	143 (33)

ADL, activities of daily living; BMI, body mass index; BNP, brain natriuretic peptide; eGFR, estimated glomerular filtration rate; HbA1c, glycated haemoglobin; HF, heart failure; IQR, interquartile range; LVEF, left ventricular ejection fraction.

Numbers with percentage are indicated in categorical variables and median with interquartile range in continuous ones.

We excluded patients who were lost at follow‐up 6 months after hospital discharge. The comorbidities have been defined in detail in our previous paper. ⁷ In brief, hypertension was diagnosed if peripheral blood pressure was >140/90 mmHg or if the patient was taking medication for hypertension. Diabetes was diagnosed when glycated haemoglobin (HbA1c) was >6.5% (standard value, Japanese Diabetes Society) or if the patient was taking medication for diabetes. Dyslipidaemia was diagnosed if total cholesterol level was >220 mg/dL, low‐density lipoprotein cholesterol level was >140 mg/dL, triglyceride levels were >150 mg/dL, high‐density lipoprotein cholesterol level was <40 mg/dL, or if the patient was on statin medication. In this study, we divided the patients into two activities of daily living (ADL) groups at discharge: independent walking outside or at home and non‐independent walking. The main drug therapy manager was defined as the person who most frequently managed the patients' drug therapy on a daily basis, that is, the patients themselves or their partners, a son or daughter, a caretaker, or a nursing home or hospital. In this study, we divided the patients into two categories as a social background based on the main drug therapy manager: patients themselves or other managers.

The primary outcome to be predicted was the incidence of all‐cause mortality during the follow‐up period. We performed follow‐ups at 6 months, 1 year, 2 years, and 3 years after discharge. Follow‐up data were collected primarily by a review of hospital records, and additional follow‐up information was obtained via telephone or mail contact with the patients or their relatives.

There is no generally accepted approach for estimating the sample size for the derivation of risk prediction models. Although we understand that it is controversial, it has been suggested to have at least 10 events per candidate variable for the derivation of a model, and this approach has been widely adopted. ¹⁶

To deal with missing variables, we performed multiple imputations to impute the missing values using the ‘missForest’ package. This imputation technique is a non‐parametric algorithm that can accommodate non‐linearities and interactions, and single‐point estimates can be generated accurately by a random forest. ¹⁷ , ¹⁸ The advantages of using the random forest model are that it can handle continuous as well as categorical responses, requires very little tuning, and provides an internally cross‐validated error estimate.

Predictors

Based on previous studies ⁸ , ⁹ , ¹⁰ and the opinions of the experts in our research group, the following potential predictors were selected: age, sex, body mass index, ADL at discharge, lifestyle (living alone, current smoker, drinking status, and management of prescribed medications), comorbidities (ischaemic heart disease, arterial fibrillation, valve disease, hypertension, chronic obstructive pulmonary disease, stroke, and active cancer), biomarkers (serum albumin, estimated glomerular filtration rate, haemoglobin, brain natriuretic peptide, and HbA1c), and echocardiographic findings (ejection fraction and left atrial dimension). The details and definitions of the variables are provided in Supporting Information, Table S2 , and all variables were defined at discharge. To ensure that the model is user‐friendly at the bedside, continuous variables in these predictor candidates were categorized as dichotomized at the point of their rounded value or commonly used ranges. Before this categorization, the association between continuous variables and outcome was investigated visually by a histogram describing the quantile of continuous predictors and frequency of outcome to confirm whether monotonic increase or decrease in mortality changed the predictor or the ‘U’ shape of the histogram (the middle range was low, and both sides were high mortality).

Statistical analysis

Patient characteristics and predictor candidates were described for each cohort. Continuous variables are described as medians and IQRs, while categorical variables are described as numbers and percentages.

The included patients were divided into two cohorts (development and test) based on calendar time to ensure that the cohorts were approximately 70% and 30%, respectively, of the total sample size. The development cohort derived from April 2015 to July 2016 was used to build the prediction model, and the test cohort from August 2016 to August 2017 was used to evaluate the prediction model. Generally, the external validation of prediction models requires different patient profiles. Therefore, this test cohort was considered appropriate for external validation because sample splitting was based on the time period, and each cohort was expected to be heterogeneous and consisted of slightly different patient profiles. ¹⁹

For selection of predictors in the development cohort, we applied the least absolute shrinkage and selection operator (LASSO) regularization with 10‐fold cross‐validation and set the optimal value of the penalty parameter (lambda.1se) using the ‘glmnet’ package. The LASSO regularization has some advantages such as choosing a few relevant variables and ignoring others to reduce the model complexity, prevent overfitting, and offer effective handling of multicollinearity. ²⁰ Subsequently, the selected predictors were set into a multivariable logistic regression model to generate the β coefficient with a standard error for each predictor. The performance of the model was evaluated based on the C index, Nagelkerke's R ² value, calibration intercept and slope, and Brier's score using the ‘rms’ package. Calibration plots were created to graphically indicate the association between the predicted and observed outcomes using locally weighted scatterplot smoothing. ²¹ We used a bootstrapping procedure (200 samples drawn with replacement from the original sample) to assess the internal validation of the model.

Subsequently, in order to perform calculations easily in clinical settings without any calculator, we developed a risk scoring system to predict the outcome using a simple integer based on each variable's β coefficient, similar to previous studies. ²² , ²³ Further, based on the sum of scores, three risk groups (low, moderate, and high) were set for a rule‐in and rule‐out approach to help in decision‐making.

For external validation, the developed model was applied to the test cohort, and the discrimination and calibration performances were calculated. The relationships between prediction and observation in each group were indicated for the calibration. In order to evaluate the additional predictive performance of these social and physical variables, we performed sensitivity analysis to develop the extra model in which these parameters were excluded and compare the predictive performance between the original and extra models using the C statistics and net‐reclassification improvement (NRI). Statistical analyses were performed between March 2021 and April 2021 using the R Version 1.1.456 (R Project for Statistical Computing) with the rms packages. Estimated values were calculated using a 95% confidence interval (CI).

Results

In the development cohort, 12 predictors were selected based on the results of the LASSO regularization (Supporting Information, Figure S2 ). Among these predictors, the calculated variable importance of the variable ‘age < 65 years’ was much smaller than that of the other variables (Supporting Information, Figure S3 ); thus, we omitted it for a more parsimonious model. Finally, 11 variables were set into the logistic regression model, and the β coefficient and standard error were calculated (Supporting Information, Table S3 ). The C statistic of the model was 0.834 (95% CI: 0.803–0.865). The bias‐corrected C statistic by bootstrapping was 0.823 (95% CI: 0.792–0.861) (Supporting Information, Table S4 ). The equation of the prediction model to calculate the probability of the outcome, other performances, and the calibration plot are shown in Supporting Information, Figure S4 . We developed the extra model in which the social and physical variables (ADL and medication management) were excluded from the original model. NRI also indicated the superiority of the original model consisted by social and physical variables (Supporting Information, Table S5 ). The calibration plot with bootstrap optimism correction showed that the prediction was well calibrated to the observation. The histograms of predicted probability and scores in the development and test cohorts were described in Supporting Information, Figures S5 and S6 , respectively.

We created a simple scoring system based on the β coefficient (Table 2 ). Further, we divided the patients into three groups, as follows: low‐risk (score 0–5), moderate‐risk (score 6–11), and high‐risk (score 12–19) groups. The observed and predicted mortality rates were described by the Kaplan–Meier curve divided by risk group in the development and test cohorts in Figure 1 .

Table 2.

Risk score

Predictor	Score
ADL (unable to walk independently)	2
Main drug management (not patients themselves)	2
Previous hospitalization for HF	2
Age
75 and over 75 years	2
85 and over 85 years	3
BMI
<25 kg/m2	2
eGFR
<30 mL/min/1.73 m2	2
Serum albumin
<4.0 mg/dL	2
<3.0 mg/dL	4
BNP
400 and over 400 pg/dL	1
600 and over 600 pg/dL	2

ADL, activities of daily living; BMI, body mass index; BNP, brain natriuretic peptide; eGFR, estimated glomerular filtration rate; HF, heart failure.

Figure 1.

The Kaplan–Meier curve divided by risk group in the development and test cohorts.

In the test cohort, the C statistic of the prediction model was 0.778 (95% CI: 0.732–0.824). The mean predicted probabilities in the groups stratified by the score were as follows: low (score 0–5), 6.9% (95% CI: 3.8–10%); moderate (score 6–11), 30.1% (95% CI: 25.4–34.8%); and high (score ≥12), 79.2% (95% CI: 72.6–85.8%) (Figure 2 and Table 3 ). The predicted probability was well calibrated to the observed outcome in both cohorts.

Figure 2.

Observation and mean predicted probability by each risk group.

Table 3.

Observation and mean predicted probability by each risk group

Group	N	Observed (%)	Predicted (%, 95% CI)
Development cohort
Low risk	9/171	5.3	6.5 (4.0–9.0)
Moderate risk	126/371	34.0	33.8 (29.8–37.8)
High risk	111/137	81.0	79.9 (74.5–85.0)
Test cohort
Low risk	9/115	7.8	6.9 (3.8–10.0)
Moderate risk	78/246	31.7	30.1 (25.4–34.8)
High risk	56/77	72.7	79.2 (72.6–85.8)

CI, confidence interval.

Low risk: score 0–6; moderate risk: score 7–11; and high risk: score 12–19. Observed: N, number of the patients with mortality/number of all the patients in each group; Observed: probability of the patients with mortality; Predicted: predicted probability for mortality.

Discussion

In this prospective registry study in Japan, we demonstrated that the KICKOFF risk score is a potential predictor of the prognosis of the incidence of all‐cause mortality after discharge among patients with AHF. Based on the score, it would be possible to predict adverse events in the high‐risk, moderate‐risk, and low‐risk groups.

This study had several strengths. First, the score included not only basic variables but also physical and social factors to predict adverse events in patients with AHF. Previous studies have shown that accurate risk scores included age, comorbidities, laboratory data, echocardiography data, and medications. ⁹ , ¹⁰ , ¹¹ , ¹² However, these scores did not contain points for physical activity or social problems. It is important to assess the physical activity and social background of patients to prevent rehospitalization for AHF. Previous studies have shown that a decline in physical performance is a valuable predictor of worse outcomes in patients with HF, ⁷ , ⁸ and social problems are associated with worse outcomes in patients with AHF. ²⁴ In this study, we developed a model that included physical activity and social background, and after the analysis, the factors of ADL at discharge (not independent walking) and social background (prescribed medications not managed by patients themselves) still remained independent predictive factors in patients with AHF. Currently, there is a higher prevalence of HF in the elderly, and the population is increasingly aging. ¹ , ² Some elderly patients still have good physical or cognitive function; in contrast, some non‐elderly patients have poor physical or cognitive function. To our knowledge, this is the first report examining the risk scores by including physical and social information that is a helpful cardiac rehabilitation and social resource for preventative approaches in patients following AHF.

Second, the score predicted the long‐term prognosis of patients with AHF. Some studies have reported risk scores in patients with CHF. ⁹ , ¹⁰ Other studies on AHF have demonstrated that the risk scores showed the prognosis only in hospitalization or short‐term observation after discharge. ¹¹ , ¹² Therefore, models of risk scores to predict the long‐term prognosis in patients with AHF have been few. Presently, patients with AHF have better outcomes than they had before because of improvements in medication, cardiac rehabilitation, or social management of HF. The time that we can spend with patients, their family members, and co‐medical staff is longer than that spent in previous clinical settings. We believe that our score will be well fitted in the present clinical setting.

Third, the score may help clinicians in easily predicting the probability of adverse outcomes and assist them in decision‐making for advance care planning (ACP) and palliative care management of patients with AHF. Based on our results, one‐third of the patients with moderate‐risk scores might have adverse events, and they and their family members should be ready to consider ACP before or after discharge. Three‐quarters of the patients with high‐risk scores might have adverse events, and they and their family members should begin to consider ACP and palliative care during their rehospitalization. In Japan, the tools for ACP and palliative care management have been recently established ²⁵ , ²⁶ ; however, there are not enough data to make them popular in patients with HF. We should be aware of the prognosis of AHF and introduce proper perspective on ACP and palliative care to patients with AHF.

Meanwhile, this study had several limitations. First, AHF diagnosis was defined by physicians using the Framingham criteria; therefore, selection or referral bias may be possible. Second, we did not have detailed information to evaluate patients' ADL using quantitative indicators, such as the Barthel index or the Functional Independence Measure score, ²⁷ , ²⁸ and cognitive functions using scales, such as the Mini‐Mental State Examination or Clinical Dementia Rating. ²⁹ , ³⁰ Third, the model may increase the risk of overfitting, which is a modelling error that fits the statistical model with too many degrees of freedom in the modelling process. Fourth, social status and physical status are subjects to change by time. Unfortunately, we did not have the status data after discharge in this registry. Importantly, this risk score could be calculated again in patients with HF at rehospitalization of HF and revised. Finally, the utility of this prediction model in other regions or countries is unclear. Japanese patients with HF with preserved efection fraction were lower BMI and had less coronary artery disease, but had a substantially higher prevalence of atrial fibrillation and lower incidence of subsequent events compared with previous Western reports. ³¹ Some treatments commonly used in Japan are not approved outside Japan. ³² In this study, we performed a temporal validation model. ³³ Therefore, we expect an evaluation for score fitting in patients with AHF in other regions or countries in the future. Further prospective research is needed to evaluate the utility and validity of our scoring system in other clinical settings.

Conclusions

In this population, the KICKOFF score may be helpful in predicting adverse events in patients with AHF over a long‐term period. Further studies are needed to prospectively validate the score in other clinical settings and to elucidate its clinical usefulness.

Conflict of interest

All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.

Funding

This research was supported by research funding from Nakajima Steel Pipe Company Limited.

Supporting information

Table S1. Patients' characteristics before missing value imputation.

Table S2. Definition of the variables.

Table S3. Beta coefficient of logistic model and Risk score.

Table S4. Performance of logistic model and bias corrected by bootstrapping.

Table S5. Extra model in sensitivity analysis.

Figure S1. Study flowchart.

Figure S2. Mean squared error and lamda in LASSO.

Figure S3. Variable coefficients in LASSO.

Figure S4. Calibration plots.

Figure S5. Histograms of predicted probability.

Figure S6. Histograms of score.

Takabayashi, K. , Okada, Y. , Iwatsu, K. , Ikeda, T. , Fujita, R. , Takenaka, H. , Kitamura, T. , Kitaguchi, S. , and Nohara, R. (2021) A clinical score to predict mortality in patients after acute heart failure from Japanese registry. ESC Heart Failure, 8: 4800–4807. 10.1002/ehf2.13664.