Current state of artificial intelligence-based algorithms for hospital admission prediction in patients with heart failure: a scoping review

Abstract

Aims

Patients with congestive heart failure (HF) are prone to clinical deterioration leading to hospital admissions, burdening both patients and the healthcare system. Predicting hospital admission in this patient group could enable timely intervention, with subsequent reduction of these admissions. To date, hospital admission prediction remains challenging. Increasing amounts of acquired data and development of artificial intelligence (AI) technology allow for the creation of reliable hospital prediction algorithms for HF patients. This scoping review describes the current literature on strategies and performance of AI-based algorithms for prediction of hospital admission in patients with HF.

Methods and results

PubMed, EMBASE, and the Web of Science were used to search for articles using machine learning (ML) and deep learning methods to predict hospitalization in patients with HF. After eligibility screening, 23 articles were included. Sixteen articles predicted 30-day hospital (re-)admission resulting in an area under the curve (AUC) ranging from 0.61 to 0.79. Six studies predicted hospital admission over longer time periods ranging from 6 months to 3 years, with AUC’s ranging from 0.65 to 0.78. One study prospectively evaluated performance of a disposable sensory patch at home after hospitalization which resulted in an AUC of 0.89 for unplanned hospital admission prediction.

Conclusion

AI has the potential to enable prediction of hospital admission in HF patients. Improvement of data management, adding new data sources such as telemonitoring data and ML models and prospective and external validation of current models must be performed before clinical applicability is possible.

Graphical Abstract

Graphical Abstract.

Introduction

Heart failure (HF) is a complex chronic clinical syndrome affecting at least 36 million people worldwide.¹ Owing to the aging population and improved treatment strategies its prevalence is increasing.² Patients suffering from HF can unexpectedly deteriorate leading to hospital admissions that burden both patients and the healthcare system.^2,3 Predicting hospital admission could lead to timely intervention, with a subsequent reduction of clinical deterioration and hospital admissions.⁴ These predictions remain challenging. Artificial intelligence’s (AI’s) capability to learn and recognize patterns from large and complex data sets could enable creating trustworthy hospital admission prediction algorithms.^5–7

Before the AI era, several methods have been described to predict hospital admission in patients with HF. Non-AI algorithms were developed using readily available variables derived in the hospital or at home using telemonitoring.^8–10 Performance of algorithms using hospital derived data remains weak, with area under ROC curve (AUC) between 0.6 and 0.7.¹⁰ Telemonitoring enables the ability to acquire many data points from a single patient at home, adding valuable information to data acquired during the sparse visits to the (outpatient) clinic.¹¹ Even though telemonitoring has made important early steps enabling home monitoring at low cost for large patient groups, previously proposed algorithms based on non-invasively derived telemonitoring data generally do not perform well enough for clinical use mainly because they result in many false positive alerts.^11–15 Invasive devices perform better but are expensive and prone to complications.¹⁶

AI-driven methods have shown great potential in many fields in healthcare including precision medicine and diagnosis, as they enable the interpretation of datasets too large or complex for human interpretation and have shown to outperform conventional approaches.^17–19 Artificial intelligence-based analysis has the potential to create preventive and predictive care for patients suffering from HF allowing intervention before symptomatic deterioration and thus possibly preventing costly hospital admission.²⁰ However, before broad and successful implementation of such analysis methods becomes possible many hurdles including technical, ethical and legal difficulties must be administered.

In this scoping review, we describe the current literature on strategies and performance of AI-based algorithms for prediction of hospital admission in patients with HF. Finally, we give, our view regarding recommendations for future research on the use of AI prediction algorithms in HF.

Methods

This systematic review was guided by the PRISMA extension for scoping reviews.²¹

Search and selection method

Three bibliographic databases (PubMed, Embase.com and Clarivate Analytics/Web of Science Core Collection) were searched for relevant literature until August 3, 2021 (Table 1). Searches were devised in collaboration with a medical information specialist (K.A.Z.). Search terms including synonyms, closely related words, and keywords were used as free-text words: ‘artificial intelligence’ and ‘heart failure’. The searches contained no date or language restrictions that would limit results to specific studies.

Table 1.

Full search query in PubMed

Search	PubMed Query August 3, 2021	Results
#3	#1 AND #2	870
#2	“Artificial Intelligence”[Mesh] OR “Artificial Intelligence*”[tiab] OR “AI”[tiab] OR “computational Intelligen*”[tiab] OR “machine intelligen*”[tiab] OR “deep learning”[tiab]
#1	“Heart Failure”[Mesh] OR “heart failure*”[tiab] OR “cardiac failure*”[tiab] OR “heart decompensat*”[tiab] OR “myocardial failure*”[tiab] OR “cardiac decompensat*”[tiab] OR “myocardial decompensat*”[tiab] OR “HFREF”[tiab] OR “diastolic dysfunction*”[tiab] OR “systolic dysfunction*”[tiab]	242 172

Articles were considered eligible if they reported retrospective or prospective data on hospital admission prediction in patients with HF as defined by the European Society of Cardiology (ESC) using AI. Traditional statistical models such as logistic regression (LR) is not included except if used as comparison to ML or data acquisition includes ML such as natural language processing(NLP). Unsupervised learning is mainly used for the identification of patterns in unlabelled data. Clustering of these patterns might correlate with increased or decreased risk of adverse events, which is not in the scope of this review.

Screening for eligibility was conducted by two individual reviewers (PC and HB). For title and abstract screening both reviewers used ASReview (University Utrecht 2021). In short, ASReview uses active learning, a type of machine learning (ML), to help researchers efficiently screen title and abstracts accelerating the process in a time with a huge information overload were searches might exceed multiple thousands of results and has proven to be an reliable asset in one to one comparisons.²² An extensive overview of the selection process using ASReview is provided in the supplementary methods.

Of all included articles, year of publication, study design, number of patients, outcome, data source, feature types, used algorithm and AUC was extracted. In addition, if reported, the confidence interval (CI), standard deviation (SD), specificity, sensitivity, true positive value (TPV), true negative value (TNV), accuracy, precision, recall, and F-score were collected and reported. In case multiple algorithms were used, the best performing algorithm was extracted together with LR for comparison. All results were described in a table and the best performing and largest cohort were described in text. Results were grouped in the following subgroups: 30-day hospital admission prediction, longer term hospital admission prediction and hospital admission prediction using telemonitoring. Meta-analysis of the selected studies was not possible due to the heterogeneity of the included studies and therefore qualitative assessment of the different algorithms predictive performance was conducted.

Results

The literature search generated a total of 2293 references: 871 in PubMed, 892 in Embase.com and 530 in Clarivate Analytics/Web of Science Core Collection. After removing duplicates of references that were selected from more than one database, 1562 references remained. After screening titles and abstracts using ASReview 34 articles remained for full-text screening. Five articles were excluded for the use of unsupervised learning, five for not including hospital admission as outcome and one for an experimental setup testing many different sub groups, models and outcomes making it impossible to extract the right information, which resulted in 23 articles included for analysis after full-text screening (Table 2).^23–45

Table 2.

Overview of included articles

Year	Authors	Title	Journal	Data and code availability
2000	Atienza, F., Martinez-Alzamora, N., et al.43	Risk stratification in heart failure using artificial neural networks	Proc AMIA Symp	Code and data not available
2016	Mortazavi, B. J., Downing, N. S., et al.32	Analysis of Machine Learning Techniques for Heart Failure Readmissions	Circ Cardiovasc Qual Outcomes	Code and data not available
2016	Turgeman, L., May, J. H.41	A mixed-ensemble model for hospital readmission	Artif Intell Med	Code and data not available
2017	Frizzell, J. D., Liang, L., et al.29	Prediction of 30-Day All-Cause Readmissions in Patients Hospitalized for Heart Failure: Comparison of Machine Learning and Other Statistical Approaches	JAMA Cardiol	Code and data not available
2017	Shameer, K., Johnson, K. W., et al.31	Predictive modelling of hospital readmission rates using electronic medical record-wide machine learning: a case-study using mount sinai heart failure cohort	Pac Symp Biocomput	Code and data not available
2018	Golas, S. B., Shibahara, T., et al.27	A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data	BMC Med Inform Decis Mak	Data available upon request, code not available
2018	Xiao, C., Ma, T., et al.30	Readmission prediction via deep contextual embedding of clinical concepts	PLoS One	Code available, data not available
2018	Mahajan, S. M., Mahajan, A. S., et al.34	Predicting Risk of 30-Day Readmissions Using Two Emerging Machine Learning Methods	Stud Health Technol Inform	Code and data not available
2018	Larburu, N., Artetxe, A., et al.45	Artificial Intelligence to Prevent Mobile Heart Failure Patients Decompensation in Real Time: Monitoring-Based Predictive Model	Mobile Information Systems	Code and data not available
2019	Allam, A., Nagy, M., et al.23	Neural networks vs. Logistic regression for 30 days all-cause readmission prediction	Sci Rep	Code and data publicly available
2019	Ashfaq, A., Sant’Anna, A., et al.25	Readmission prediction using deep learning on electronic health records	J Biomed Inform	Saple of the data and code available
2019	Awan, S. E., Bennamoun, M., et al.28	Machine learning-based prediction of heart failure readmission or death: implications of choosing the right model and the right metrics	ESC Heart Fail	Data available after approval, code available
2019	Awan, S. E., Bennamoun, M., et al.26	Feature selection and transformation by machine learning reduce variable numbers and improve prediction for heart failure readmission or death	PLoS One	Code available, data available upon request
2019	McKinley, D., Moye-Dickerson, et al.35	Impact of a Pharmacist-Led Intervention on 30-Day Readmission and Assessment of Factors Predictive of Readmission in African American Men With Heart Failure	Am J Mens Health	Code and data not available
2019	Mahajan, S. M., Ghani, R.38	Using Ensemble Machine Learning Methods for Predicting Risk of Readmission for Heart Failure	Stud Health Technol Inform	Code and data not available
2019	Mahajan, S. M., Ghani, R.33	Combining Structured and Unstructured Data for Predicting Risk of Readmission for Heart Failure Patients	Stud Health Technol Inform	Code and data not available
2020	Angraal, S., Mortazavi, B. J., et al.24	Machine Learning Prediction of Mortality and Hospitalization in Heart Failure With Preserved Ejection Fraction	JACC Heart Fail	Code and data not available
2020	Chu, J., Dong, W., et al.36	Endpoint prediction of heart failure using electronic health records	J Biomed Inform	Code available, data not available
2020	Chen, P., Dong, W., et al.37	Interpretable clinical prediction via attention-based neural network	BMC Med Inform Decis Mak	Code not available, data available upon request
2020	Desai, R. J., Wang, S. V., et al.39	Comparison of Machine Learning Methods With Traditional Models for Use of Administrative Claims With Electronic Medical Records to Predict Heart Failure Outcomes	JAMA Netw Open	Code and data not available
2020	Stehlik, J., Schmalfuss, C., et al.44	Continuous Wearable Monitoring Analytics Predict Heart Failure Hospitalization: The LINK-HF Multicentre Study	Circ Heart Fail	Code and data not available
2021	Lewis, M., Elad, G., Beladev, M., et al.40	Comparison of deep learning with traditional models to predict preventable acute care use and spending among heart failure patients	Sci Rep	Code available, data on request
2021	Ben-Assuli, O., Heart, T., et al. 42	Profiling Readmissions Using Hidden Markov Model-the Case of Congestive Heart Failure	Information Systems Management	Code and data not available

Thirty-day hospital readmission prediction

Sixteen articles reported performance of AI algorithms for 30-day all-cause and/or HF hospital readmission prediction with AUC ranging from 0.61 up to 0.79. Results from data extraction are summarized in Table 3. Ben-Assuli et al.⁴² reported the highest AUC with 0.79, using a Hidden Markov Model on data on 4661 adult HF patients with at least five in- or outpatient clinic visits. Thirty-eight time-varying covariates were retrospectively derived from 6 hospital information systems including a comprehensive electronic health record (EHR), laboratory information management system and picture archiving and communication system (PACS) for imaging results. The algorithm’s performance enhanced when data from more clinical visits were present, resulting in an AUC of 0.79 after four visits for all patients. Patients were divided in three states depending on readmission risk. The model performed worst in the most vulnerable group and best in the medium risk group with an AUC after four visits of 0.65 and 0.77, respectively.⁴² Shameer et al. reported an algorithm with similar performance, using retrospective EHR data of 1068 HF patients, in whom more than 4000 features were extracted from the EHR including diagnosis codes, medication, laboratory measurements and vital signs. Correlation-based feature selection of EHR data was performed and a naïve bayes classifier for readmission prediction which resulted in an AUC of 0.78.³¹ Medications yielded the most predictive value. The study with the largest patient group was by Allam et al. who included 273 000 patients in their analysis retrieved of an administrative insurance claims dataset including socio-demographics, hospital admission event, comorbidity, and diagnosis. The authors best performing deep learning (DL) algorithm utilized a recurrent neural network (RNN) combined with conditional random fields, which resulted in an AUC for 30-day readmission prediction of 0.64.²³ Penalized logistic regression (LASSO) had a similar AUC of 0.64 (Table 4). Prediction algorithms based on EHR data generally outperform algorithms based on data from national databases or insurance data and only EHR-based algorithms achieved moderate performance (Table 3).

Table 3.

Results for 30-day prediction models

Year	Authors	Design	No. patients	Outcome (30-day readmission)	Data source	Data used	Model used	AUC	Performance metrics reported
2016	Mortazavi, B. J., Downing, N. S., et al.32	Retrospective	977	HF	EHR + Telemonitoring	472 features including clinical data, physical examination, laboratory, demographics, socio-economics	Random forest	0.68 (CI: 0.670–0.678)	PPV 0.22, Sens 61%, Spec 61%, F-score 0.32
2016	Turgeman, L., May, J. H.41	Retrospective	4840	All-cause	EHR	Demographics, vitals, laboratory, comorbidities	C5 SVM ensemble	0.7	TH 0.7 Sens 26%, spec 91%, PPV 0.260, NPV 0.911, F1 0.261
2017	Frizzell, J. D., Liang, L., et al.29	Retrospective	56477	HF	Insurance	Demographics, socio-economic status, medical history, medication, vital signs, laboratory, interventions	Tree augmented naive bayesion network	0.62
2017	Shameer, K., Johnson, K. W., et al.31	Retrospective	1068	All-cause	EHR	ICD-codes, medication, laboratory, vital signs, procedures	Naive Bayes	0.78	Accuracy 83,19%
2018	Xiao, C., Ma, T., et al.30	Retrospective	5393	All-cause	EHR	Comorbidity, laboratory, medication,	Gated recurrent unit RNN	0.61 (SD 0.0153)	PR-AUC 0.39, accuracy 0.6934
2018	Golas, S. B., Shibahara, T., et al.27	Retrospective	11510	All-cause	EHR	Demographic and clinical data (comorbidity, loboratory, medication, procedures), notes	Deep Unified Network	0.71	Accuracy 0.646, Precision 0.360, recall 0.652, f1 0.464
2018	Mahajan, S. M., Mahajan, A. S., et al.34	Retrospective	1778	All-cause	EHR	Laboratory, vitals, demographics	Boosted trees	0.719
2019	Allam, A., Nagy, M., et al.23	Retrospective	272.778	All-cause	National database	socio-demographics (age/gender), Pay-source, hospital adission events, diagnosis, procedures	RNN with conditional random fields	0.642 (CI 0.640–0.645)	TPV 0.57, FPV 0.37
2019	Awan, S. E., Bennamoun, M., et al.	Retrospective	10757	HF	Nation-wide system(EHR, GP etc)	demographics, admission characteristics, medical history, socio-economics, medication history	Multi-layer perception	0.66	Sens 56%, spec 66%
2019	Mahajan, S. M., Ghani, R.	retrospective	1629	All-cause	EHR	Laboratory, vitals, demographics, clinical notes	NLP for unstructured data, LR for prediction	0.65(CI 0.59-0.71)
2019	Awan, S. E., Bennamoun, M., et al.	Retrospective	10757	HF	Nation-wide system (EHR, GP, and so on)	demographics, admission characteristics, medical history, socio-economics, medication history	Multi-layer perception	0.63	Sens 48%, spec 70%
2019	Mahajan, S. M., Ghani, R.	Retrospective	36245	All-cause	EHR	Demographics, vitals, laboratory, comorbidities	ExtraTrees	0.70 (CI 0.69–0.71)	F1 score 0.58
2019	Ashfaq, A., Sant’Anna, A., et al.	Retrospective	7655	All-cause	EHR	Demographics, comorbidity, admission type, medication, all procedures, laboratory results, ICD-codes and more	RNN with LSTM	0.77 (SD: 0.005)	F1 score 0.51
2019	McKinley, D., Moye-Dickerson, P., et al.	Retrospective	132	All-cause	EHR	Demographic, social, and clinical data	K nearest neighbour	0.768	Accuracy 0.77, FPV 0.214, FNR 0.250
2021	Ben-Assuli, O., Heart, T., et al.	Retrospective	4661	HF	EHR	Demographics, laboratory, medical imaging, comorbidity,	Hidden Markov Model	0.79	Accuracy 0.89

EHR, electronic health record; PPV, positive predictive value.

Table 4.

Performance of logistic regression versus artificial intelligence

Year	Authors	Model used	AUC AI	AUC LR
2016	Mortazavi, B. J., Downing, N. S., et al. 32	Random forest	0.68 (CI: 0.670–0.687)	0.54 (CI: 0.563–0.550)
2016	Turgeman, L., May, J. H. 41	C5 ensemble	0.70	0.70
2017	Frizzell, J. D., Liang, L., et al. 29	Tree augmented naive bayesion network	0.62	0.62
2018	Xiao, C., Ma, T., et al.30	Gated recurrent unit RNN	0.61 (SD: 0.013)	0.59 (SD: 0.012)
2018	Mahajan, S. M., Mahajan, A. S., et al.34	Boosted trees	0.72	0.62
2018	Golas, S. B., Shibahara, T., et al.27	Deep Unified Network	0.70	0.66
2019	Awan, S. E., Bennamoun, M., et al.26	Multi-layer perception	0.63	0.55
2019	Allam, A., Nagy, M., et al.23	RNN with conditional random fields	0.64 (SD: 0.0027)	0.64 (SD: 0.0028)
2019	Mahajan, S. M., Ghani, R.38	ExtraTrees	0.70	0.70
2020	Chen, P., Dong, W., et al.37	Attention-based model	0.69 (SD: 0.047)	0.68 (SD: 0.039)
2020	Angraal, S., Mortazavi, B. J., et al.24	Random forest	0.76 (CI: 0.71–0.77)	0.73 (CI: 0.67–0.79)
2020	Desai, R. J., Wang, S. V., et al.39	GBM	0.78 (CI: 0.753–0.802)	0.74 (CI: 0.711–0.766)
2021	Lewis, M., Elad, G., et al.40	Sequential deep learning	0.78 (CI: 0.784–0.790)	0.75 (CI: 0.744–0.751)

Long-term hospital admission prediction

In total, six studies predicted hospital (re)admission for a time period longer than 30 days (Table 5). Five studies used retrospective data and one used prospective data. Outcomes ranged from 6-month readmission to 3-year readmission. All but one reported AUC ranging from 0.65 to 0.78. The best performing two algorithms both reported an AUC of 0.78. Lewis et al.⁴⁰ used insurance claims retrospective data of 92 000 patients, divided in knowledge-driven and data-driven features using NLP, for predicting 6-month all-cause admission. The authors reported that DL networks, a convolutional neural network with LSTM in this case, outperforms traditional ML models (feed forward neural network and LR). Moreover, sequential DL models outperformed non-sequential DL models. The second algorithm-derived data from both EHR and insurance claims predicting 1-year HF hospitalization.³⁹ A total of 54 variables were collected from the claims including demographics, no. of hospitalizations, medication and comorbidity, and eight variables from the EMR laboratory and left ventricular EF. The authors utilized multiple different models of which a gradient boosted model (GBM) performed best with an AUC of 0.78 (compared with 0.75 when only claims data was used). The longest prediction period (3 years) utilized a random forest classifier resulting in an AUC of 0.76 for admission prediction.²⁴ This result was achieved using data from a trial including patients with HF with preserved ejection fraction including of baseline demographic and clinical data, ECG, laboratory data, and questionnaires.

Table 5.

Results for long-term hospital admission prediction

Year	Authors	n. patients	Design	Outcome	Data source	Data used	Model used	AUC
2016	Mortazavi, B. J., Downing, N. S., et al.	977 HF patients	Retrospective	180-day HF readmission	EHR + Telemonitoring	472 features including clinical data, physical examination, laboratory, demographics, socio-economics	Random forest in to SVM	0.657 (CI: 0.652–0.661)	PPV 0.51, Sens 0.95, spec 0.15, F-score 0.66
2020	Chen, P., Dong, W., et al.	736 HF patients	Retrospective	1-year all-cause readmission	EHR	105 features including demographics, vital signs, laboratory, echocardiography, comorbidities, medication	Attention-based model	0.691 (SD: 0.047)	Acc 0.667, Precision 0.710, Recall 0.795, F1 0.749
2020	Angraal, S., Mortazavi, B. J., et al.	1767 HFpEF patients	Retrospective	3-year HF admission	EHR	Demographics, clinical data, ECG, laboratory, questionnaires	Random forest	0.76 (CI: 0.71–0.85)
2020	Desai, R. J., Wang, S. V., et al.	9502 HF patients	Prospective	1-year HF admission	EHR + Insurance	54 variables including ICD score, ICD/CRT-D, HF related medication, frailty index, laboratory, BMI, ejection fraction	GBM	0.78 (CI: 0.753–0.802)
2021	Lewis, M., Elad, G., Beladev, M., et al.	92260 HF patients	Retrospective	6-month all-cause admission, 6-month ED admission	Insurance	demographics, episode counts and trends, hospital length of stay, readmission rates, costs, comorbidity, procedures, medication, machine derived features using NLP.	Sequential deep learning	0.78 (CI: 0.784–0.79)
2000	Atienza, F., Martinez-Alzamora, N., et al.	132 HF patients	Retrospective	1-year admission	EHR	Demographic, clinical, non-invasive laboratory	Neural Network	Accuracy 93.2% (AUC not reported)	sens 81%, Spec 94%
2020	Chu, J., Dong, W., et al.	25402HF patiënts	retrospective	HF readmission	EHR	demographics, medication, procedures, diagnosis, laboratory, vital signs, physical examination, etc.,	RNN for data processing, LR for endpoint prediction	0.644 (SD: 0.005)

Telemonitoring for hospitalization prediction

Three studies used AI algorithms on telemonitoring data for hospital admission prediction (Table 6). Stehlik et al.⁴⁴ monitored one hundred subjects prospectively at home for 3 months using a disposable multisensory patch monitoring vital parameters including ECG (arrhythmia burden, heart rate variability), skin impedance, temperature, and so on. The data acquired by the device was continuously uploaded through a smartphone. Patients were instructed to wear the device 24 h a day. After initial placement of the device, baseline patterns in the data are identified. When this baseline is finalized, the device switched to surveillance mode that compared temporal data to the baseline patterns using similarity-based modelling. This approach resulted in an AUC of 0.89 for HF readmission and 0.84 for unplanned non-trauma hospital admission. Sensitivity for HF hospitalization was 86% at a specificity of 87.5%. The remaining two studies used retrospective data both derived from clinical trials in which eHealth was used,^32,45 which resulted in and AUC of 0.68 and 0.67 utilizing a random forest and Naïve Bayes, respectively.

Table 6.

Results for hospitalization prediction using telemonitoring

Year	Authors	No. of patients	study design	Outcome	Data source	Data used	Model	AUC	Additional performance metrics
2016	Mortazavi, B. J., Downing, N. S., et al.32	977 HF patients	Retrospective	30-day HF readmission, 180 day HF readmission	EHR + Telemonitoring	472 features including clinical data, physical examination, laboratory, demographics, socio-economics	Random forest	0.678 (CI: 0.670–0.678)	PPV 0.22, Sens 0.61, Spec 0.61, F-score 0.32
2020	Stehlik, J., Schmalfuss, C., et al.44	100 HF patients	Prospective	HF hospitalization	Wearable sensor patch	Wearable sensory patch with ECG (heart rate variability, arrythmia), skin impendance (respiratory rate), skin temperature, activity, posture	Similarity-based modelling	HF 0.86, All-cause 0.80	HF hospitalization sens 76, spec 85, All-cause sens 69, spec 85
2018	Larburu, N., Artetxe, A., et al.45	241 HF patients	Retrospective	HF decompensation (readmission + home intervention)	EHR + Telemonitoring	Telemonitoring (vital signs, weight, questionnaires), baseline (demographics, comorbitity, laboratory)	Naive Bayes	0.67	Sens 0.76, 28.64 FA/py

Discussion

This scoping review evaluated the current performance of AI-based hospital (re-)admission prediction algorithms based on EHR, administrative and telemonitoring data, and resulted in the following main findings. (i) Multiple different classifiers were proposed all using different sets of data to predict hospital admission for patients with HF making it challenging to accurately compare results. (ii) AI methods using EHR or administrative data achieved moderate performance, one AI method using telemonitoring data achieved good performance; (iii) EHR data generally performed better in comparison to administrative data or research data which is consistent with clinical expectation since EHR data yields much more comprehensive clinical information; and (iv) almost all studies were performed on retrospective data without external or prospective validation.

Although some AI-models seem promising, only the AUC was reported in all studies and additional performance metrics were only sometimes reported. A more detailed evaluation is needed to provide comprehensive insight in the performance of a model. Machine learning algorithms are prone to overfitting, meaning the algorithm is trained extensively within a population. As a consequence, this can result in a model that performs well within this population, but not outside of this population, making generalizability of the results questionable. Moreover, many articles proposed a novel method and compared this with existing methods. We chose to only report the best performing algorithm which always is the proposed method. However, it is questionable if the same amount of feature engineering and/or hyperparameter optimization was performed on the compared method as to the proposed method, making the best performing method more prone to overfitting. Overfitting can be minimized by using appropriate validation methods such as nested cross validation and tested with external or prospective validation.⁴⁶ The risk of overfitting is a major limitation of all included articles.

Hospital (re-)admission of HF patients is a major problem resulting in high patient disease burden and health care expenses.^2,3 One proven effective strategy for reducing hospital admission in this patient group is through signalling pre-clinical deterioration using invasive pulmonary artery pressure home monitoring, which yielded a 37% reduction in hospitalization.^4,16 This indicates that detection of early, pre-clinical, deterioration can indeed result in a reduction of hospital admissions. Randomized controlled trials on the effect of well performing, prospectively validated, AI-based hospital admission prediction reduction are needed to further evaluate if and to what extend early deterioration prediction leads to prevention of hospital (re-)admission and subsequent disease burden and cost.

Similar moderate performance of AI-hospitalization-prediction algorithms were achieved in non-HF patients. For example, Rajkomar et al.⁴⁷ utilized a deep neural network (DNN) in a cohort of 216,221 adult patients hospitalized for more than 24 h for any cause and reported an AUC of 0.75–0.76 for 30-day hospital readmission using EHR data including notes. Hilton et al.⁴⁸ reported an AUC of 0.76 for prediction of 30-day readmission using a DNN trained, validated and tested on 708 089 any cause hospitalized patients using EHR data. Interestingly, Morawski et al.⁴⁹ predicted 6-month all-cause hospital admission for all patients aged older than 18 years in the EHR including outpatient clinic patients of a large care facility in Massachusetts with an AUC of 0.84. Translating this to our HF population, it might be possible to predict hospital admission in outpatient clinic patients which increases the population leading to the prevention of more hospital admissions than when predicting readmission alone.

One of the questions of using AI for clinical decision support is whether it is able to outperform traditional statistical methods such as LR. A systematic review published in 2019 including 71 studies that compared ML with LR concluded that ML generally did not result in better performance than LR.⁵⁰ Risk of bias was found to be higher in the studies that did report higher AUC for ML in comparison to LR. However, this review used articles published before 2017. Our results show that ML achieves better performance in terms of AUC in most studies but confidence intervals overlap in most cases. However, the performance of ML increases over time. If this increase of performance will continue in the near future with further advances made in model development ML might be able to structurally outperform LR for this purpose (Table 5).

Future perspective

In the future, connecting multiple conventional, readily available (e.g. clinical variables, imaging data etc.) and newer data sources (health trackers, social media behaviour, genomic sequencing) could add to a bright future by exposing new preventive, diagnostic and therapeutic advantages. For example, prediction models may not only predict deterioration but also to predict and thus suggest effect of different treatment strategies further preventing patient deterioration In theory, this would prevent hospitalization, enable personalized medication advice, and provide insight into who needs an outpatient clinic visits, making it possible to scale down given care until suggested differently. In summary, prediction models could enhance scalability and cost-effectiveness of our healthcare system.

In order to get to the point of AI-driven clinical support systems advising doctors and patients daily and thus transforming current reactive ‘disease care’ towards proactive personalized and predictive medicine, substantial development of such models is still required. To enable effective use of multiple data sources (such as EHR, medication prescription systems, telemonitoring platforms and more), ‘FACT’ and ‘FAIR’ principles might be important to take in to consideration. FACT and FAIR are acronyms for ‘Fairness, Accuracy, Confidentiality, and Transparency’ and ‘Findable, Accessible, Interoperable, and Re-usable’ respectively.⁵¹ Moreover, researchers and developers must find a way to work with data given that privacy is guaranteed in accordance with all the regulations. Before broad implementation can be pursued aspects as patient benefit and cost-effectiveness can be evaluated. In addition, a cultural change must be accomplished meaning both patient and health care provider will become more familiar with such systems to efficiently change the current workflow. If this can all be accomplished, AI-based clinical support systems have the potential to positively change HFcare.

Limitations

This review has several limitations. Included studies are extensively heterogenetic in terms of population, data types, and their representation and used model making comparison challenging. Moreover, the aim of studies differed between studies. Some aimed to develop an prediction algorithm with state of the art results whereas others aimed to compare different technical strategies to evaluate if they can enhance performance in a small data set. We did not conduct a search in Google Scholar which may have led to missed relevant articles. For eligibility assessment, we used ASReview which is quite new and not yet broadly implemented. However, ASReview is based on active learning which is extensively reviewed and adoption of ML-based selection methods are used more and more. Moreover, one by one screening by humans is far from perfect leading to 10% missed articles.⁵² To reduce the chance of relevant missed articles, two separate reviewers conducted the full selection protocol which did not lead to largely different results.

Conclusion

In conclusion, AI has the potential to play an important role in early prediction of clinical deterioration in HFpatients. Current AI algorithms use EHR or telemonitoring data, and resulting in moderate to good prediction models. However, lack of external and prospective validation, resulting in a high risk of overfitting, is a major limitation to all studies. Prospective studies, with external validation are warranted to improve current models and validate their performance.

Supplementary material

Supplementary material is available at European Heart Journal – Digital Health.

Funding

University of Amsterdam Research Priority Agenda Program AI for Heath Decision-Making.

Data Availability

No new data were generated or analyzed in this review.