Development and validation of a neural network survival prediction model for ischemic heart disease

Abstract

Background

Current risk prediction models for ischemic heart disease in clinical use are relatively simple and use a limited collection of well-known risk factors. Using machine learning to integrate a broader panel of features from electronic health records (EHRs) may improve post-angiography prognostication.

Methods

This retrospective model development and validation study was based on Danish EHR data. Icelandic EHR data were used for external test. Patients with a coronary angiography-confirmed diagnosis of coronary atherosclerosis between 2006 and 2016 were included for model development (n = 39,746). Time to all-cause mortality, the prediction target, was tracked until 2019, or up to 5 years, whichever came first. To model time-to-event data and deal with censoring, neural network-based discrete-time survival models were used. The model, PMHnet, uses 584 different features including clinical characteristics, laboratory tests, and diagnosis and procedure codes. Model performance was evaluated using time-dependent AUC (tdAUC) and the Brier score. PMHnet was benchmarked against the updated GRACE2.0 risk score and less feature-rich neural network models. Models were evaluated using hold-out data (n = 5000) and external validation data from Iceland. Feature importance and model explainability were assessed using SHAP analysis.

Results

On the test set (n = 5000), the tdAUC of PMHnet was 0.88 [ 0.86–0.90] (case count = 196) at six months, 0.88 [0.86–0.90] (cc = 261) at one year, 0.84 [0.82–0.86] (cc = 395) at three years, and 0.82 [0.80–0.84] (cc = 763) at five years. PMHnet showed similar performance in the Icelandic data. Compared to the GRACE2.0 score and intermediate models limited to GRACE2.0 features or single data modalities, PMHnet had significantly better model discrimination across all evaluated prediction timepoints.

Conclusions

More complex and feature-rich machine learning models can better predict all-cause mortality in ischemic heart disease and may be used by clinicians and patients to inform and guide treatment and management.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12933-026-03078-3.

Introduction

In patients with ischemic heart disease, improved prognostication by clinical application of the wide array of known but presently unapplied prognostic risk factors and disease markers may serve a key function in tailoring precision medicine for the individual patient [1–4]. The clinical demand for prognostication tools is underscored by a class IIa recommendation for assessing risk of patients with non–ST-elevation myocardial infarction (nSTEMI) by the updated version of the Global Registry of Acute Coronary Events (GRACE) score (GRACE2.0) in the 2023 European Society of Cardiology (ESC) guidelines [2]. However, GRACE2.0 and other traditional risk scoring schemes in ischemic heart disease such as Framingham, TIMI, and SCORE2.0 use a limited set of input features (< 10) and likely grossly underutilize data on risk factors readily available in modern electronic health records (EHRs) [5–8].

Integrating a richer set of input features could be overcome with machine learning (ML) models such as neural networks. These can capture non-linear interactions without the need for imputation of missing data or expert feature engineering [9, 10], leveraging the multitude of heterogeneous healthcare data stored in EHRs and national registries in the development of clinical decision support tools. ML-based approaches have shown promising results for risk-estimation in ischemic heart disease with better performance than traditional models [11–15]. In patients with stable ischemic heart disease, Motwani et al. showed that an ML algorithm combining clinical variables with imaging variables from coronary CT angiography predicted 5-year all-cause mortality better than traditional models based on clinical metrics alone [11]. Similarly, Mohammad et al. developed and validated a neural network model for prediction of 1-year mortality and re-admission for heart failure after incident myocardial infarction with greater discrimination than the GRACE2.0 score [13]. While these models represent important additions to the field of ML-based prognostication in ischemic heart disease, most ML-based approaches in the domain treat mortality prediction as a binary classification problem and do not include time-to-event information nor censored observations. Besides possibly limiting performance, this effectively prevents a model from distinguishing between mortality after two weeks and after two years, which is of obvious clinical interest.

Here, we describe the development and validation of a novel neural network-based time-to-event model for prediction of mortality in patients with ischemic heart disease after a coronary angiography-confirmed diagnosis of coronary artery disease, here defined as one-, two-, or three vessel-disease or diffuse coronary atheromatosis. Our model, PMHnet, is a prognostic model that operates with a prediction horizon of five years and provides individualised, explainable, and time-dependent probabilistic estimates for survival from 584 different features extracted from population-wide healthcare registries and complete EHRs.

Methods

Overview of data sources

The PMHnet model was developed using a derivation cohort defined from the Danish National Patient Registry (NPR) and the Eastern Danish Heart Registry (EDHR) [16]. The EDHR contains structured information on all coronary artery angiographies performed in the Capital Region of Denmark and Region Zealand. The cohort was linked to a population-wide EHR database that covers Eastern Denmark from 1st of January 2006 to the 7th of July 2016 (BTH) and genotype data from the Copenhagen Hospital Biobank Cardiovascular Diseases study (CHB-CVD) [17–19]. The BTH dataset fully covered Eastern Denmark (2.6 million patients). Outcomes were obtained from the Central Person Registry and the Danish Register for Causes of Death [20, 21]. Data sources were linked using encrypted Danish personal identification numbers [21].

Cohort inclusion criteria

First, we identified all adult Danish citizens (> 18 years of age) in NPR with an ICD-10 code for suspected or confirmed ischemic heart disease (I20-I25) who had undergone their first diagnostic work-up for ischemic heart disease, including a routine invasive coronary angiography between January 1, 2006, and June 1, 2016 which after assessment for the presence of atherosclerotic changes in the left anterior descending artery (LAD), the left circumflex artery (LCx) and the right coronary artery (RCA), demonstrated one-, two-, or three-vessel disease (1–3VD) or diffuse atheromatosis (DIF). Vascular disease is here defined as a diameter stenosis above 50% [22], and if atherosclerotic changes were present, but below 50%, it was termed diffuse atheromatosis. For patients fulfilling these criteria (n = 39,746), we used the date of the coronary angiography as the index date and included up to five years of follow-up; patients were followed until either death or censoring, whichever came first. Using the hold-out method, the derivation data were randomly divided into a training set (n = 34,746) and a test set (n = 5000) used for model development and independent assessment of performance, respectively [23] (Fig. 1, Table 1).

Fig. 1.

Flowchart of inclusion of patients with ischemic heart disease in the training and test sets. The flowchart shows how the derivation cohort was identified based on data from NPR and EDHR. CAG: Coronary arteriography. EHRs: Electronic health records. EDHR: Eastern Danish Heart Registry. ICD-10: International Statistical Classification of Diseases and Related Health Problems, 10th revision. IHD: Ischemic heart disease. NPR: Danish National Patient Registry

Table 1.

Cohort characteristics for training, test, and external validation set

	Training (n = 34,746) Danish population	Test (n = 5000) Danish population	Validation (n = 8288) Icelandic population
General characteristics
Male sex	23,413/67.3%	3410/68.2	5876/70.9%
Age (year) [95%CI]	66.0 [65.7;66.4]	66.2 [66.0;66.3]	66.0 [65.8; 66.2]
Height (cm) [95%CI] (NA)	172.8 [172.7;172.9] (6.5%)	173 [172.7;173.2] (6.3%)	174.3 [174.1; 174.5]
Weight (kg) [95%CI] (NA)	81.3 [81.1;81.5] (4.3%)	81.7 [81.1;82.1] (4.1%)	87.2 [86.8; 87.5]
Comorbidities (ICD10)
Diabetes	4635/13.3%	694/13.9%	1035/12.5%
Hypertension	788/2.27%	105/2.1%	216/2.6%
COPD	2586/7.44%	389/7.78%	715/8.6%
Dyslipidemia	4291/12.3%	641/12.8%	381/4.6%
Medication (ATC)
Lipid-lowering drugs	18,188/52.3%	2679/53.6%	6592/79.5%
Anti-hypertensive drugs	25,684/73.9%	3679/73.6%	7494/90.4%
Type−2 diabetes drugs	5575/16.0%	808/16.2%	1048/12.6%
Insulin	2088/6.01%	320/6.4%	158/1.9%
P-creatinine,mmol/l [95%CI] (NA) (*)	84.8 [84.5;85.2] (10.1%)	84.2 [83.4;85.1] (10.8%)	91.7 [90.6; 92.9] (8.3%)
P-creatinine NPU04998 (disc.)    Within ref. range    Above ref. range    Data missing	15,490/44.6% 4,437/12.8% 14,819/42.6%	2,188/43.8% 659/13.2% 2,153/43.1%
B-basophiles NPU01349 (disc.)    Within ref. range    Above ref. range    Data missing	12,982/37.4% 192/0.56% 21,572/62.1%%	1,816/36.3% 28/0.56% 3,156/63.1%
P-basic phosphatase NPU27783 (disc.):    Within ref. range    Above ref. range    Data missing	12,597 / 36.3% 1,783/5.13% 20,366/58.6%	1,737/34.7% 280/5.6% 2,983/59.7%
Elevated cardiac biomarkers (CKMB, TnI, or TnT) (NA)	15,531/44.7% (38.7%)	2158/43.2% (40%)	1599/19.3% (39.6%)
Abnormal EKG:    Yes    No    Data missing	14,679/42.2% 10,64/30.6% 9,426/27.1%	2,043/40.9% 1,554/31.1% 1,403/28.1%	–
Familial IHD:    Known family history    No known family history    Data missing	10,558/30.4% 18,067/52.0% 6,121/17.6%	1,495/29.9% 2,655 /53.1% 850 /17%	–
Heart rate, bpm [95%CI] (NA)	74.7 [74.5;74.9] (20.2%)	74.2 [73.7;74.8] (21.1%)	69.6 [69.3; 70.0] (23.7%)
Blood pressure   Systolic, mmHg [95%CI] (NA)   Diastolic, mmHg [95%CI] (NA)	139.0 [138.7;139.2] (17.6%) 77.9 [77.7;78.1] (33.5%)	138.9 [138.2;139.7] (17.9%) 77.7 [77.2;78.2] (34%)	148.1 [147.6; 148.6] (14.0%) 85.5 [85.2; 85.8] (14.0%)
Cardiac arrest at admission?	513/1.5%	69/1.4%	130/1.6%
ICD or PM	557/1.6%	88/1.8%	188/2.3%
Killip class
Killip: 1	14,297/41.1%	2085/41.7%	3469/41.9%
Killip: 2	455/1.3%	62/1.2%	1028/12.4%
Killip: 3	138/0.4%	13/0.3%	191/2.3%
Killip: 4	170/0.5%	22/0.4%	135/1.6%
Data missing	19,686/56.7%	2818/56.4%	3465/41.8%
Left-ventricular ejection fraction (%)  LVEF: [0,10]  LVEF: (10,20]  LVEF: (20,30]  LVEF: (30,40]  LVEF: (40,50]  LVEF: (50,60]  Data missing	100/0.3% 675/1.9% 1305/3.8% 1760/5.1% 3915/11.3% 11,068/31.9% 15,923/45.8%	16/0.3% 78/1.6% 171/3.4% 258/5.2% 530/10.6% 1576/31.5% 2371/47.4%	– 43/0.5% 166/2.0% 308/3.7% 758/9.1% 2175/26.3% 4838/58.4%
Smoking status
Active smoker	10,237/29.5%	1497/29.9%	1551/18.8%
Former smoker	11,961/34.4%	1758/35.2%	4331/52.2%
Never smoked	9347/26.9%	1296/25.9%	2316/28.0%
Data missing	3201/9.2%	449/9.0%	101/1.2%
Coronary pathology
Diffuse atheromatosis	9288/26.7%	1331/26.6%	652/7.9%
1 vessel disease	13,310/38.3%	1936/38.7%	2740/33.0%
2 vessel disease	6469/18.6%	945/18.9%	1424/17.2%
3 vessel disease	5679/16.3%	788/15.8%	1133/13.6%
Data missing	–	–	28.37% (#)

The comorbidities are defined from the ICD-10 codes that had been assigned to a given patient prior to the index date. Medication is defined from prescriptions prior to index date. Lipid-lowering drugs: C10, anti-hypertensive drugs: C02, C03, C07, C08 and C09, type−2 diabetes drugs: A10B, insulin: A10A. 95%CI: 95% confidence intervals. ATC: Anatomical Therapeutic Chemical Code. ICD-10: International Statistical Classification of Diseases and Related Health Problems, 10th Revision. NA: Not applicable

(*): See supplementary methods for details on differences between continuous and discrete classification of creatinine

(#): Diffuse atheromatosis could not be defined with complete certainty for 28.4% of the Icelandic data, and coronary pathology was therefore set as NA in such cases

COPD: Chronic obstructive pulmonary disease, ICD: Implantable cardioverter-defibrillator, LVEF: Left-ventricular ejection fraction. PM: Permanent pacemaker

Baseline characteristics and statistics

For the cohort characteristics presented in Table 1, comorbidities were defined from the ICD-10 codes assigned before or at the index date: diabetes was defined as E10 or E11; hypertension was defined as I10, I11, I12, I13, I14, or I15; COPD was defined as J44; and dyslipidemia was defined as E78.0, E78.1, E78.2, E78.3, E78.4, E78.5, and E78.9. The medication use was defined from prescriptions redeemed prior to or at the index date, using the same definition as used in Kiiskinen et al. [24], that is: lipid-lowering drugs is ATC class C10; anti-hypertensive drugs is C02, C03, C07, C08, and C09; type−2 diabetes drugs is A10B; and insulin is A10A. For continuous features (age, height, etc.) the mean is given along with 95% bootstrap confidence intervals (CI) and, if applicable, the amount of missingness in parentheses—mean [low, high] (missingness). The mean and CI are calculated using only the non-missing features. For categorical features the raw counts and relative frequencies are both specified.

Feature inclusion and general pre-processing

For each patient we extracted the following data: all diagnosis codes (ICD-10) and procedure codes (SKS/NOMESCO) assigned to at least 1% of the cohort between 1st of January 1994 and the time of the coronary angiography (from NPR); all results from routine in-hospital laboratory tests taken on at least 5% of the cohort between 5 years prior-to and up until the time of the coronary angiography (from BTH); 23 other clinical features such as sex, age, smoking status, coronary pathology, etc. (from EDHR + BTH). The resulting 584 features belongs to five different general categories: ClinicalOne (8 features), ClinicalTwo (15 features), Diagnoses (322 features), Procedures (154 features), and Biochemical (85 features) (Table 2 and Supplementary Data 1). The ClinicalOne category included the same eight input features as used by GRACE2.0 and ClinicalTwo had 14 additional clinical features that were selected based on availability. If multiple entries for a given variable were available (e.g. multiple blood pressure measurements), we used a last-value-carried-forward approach.

Table 2.

Input features used for model development

Category	Features
ClinicalOne (GRACE2.0)	Age, pulse, systolic blood pressure, cardiac arrest at presentation (yes/no), abnormal cardiac enzymes (yes/no), Killip-class, creatinine, ST-segment deviation (yes/no)
ClinicalTwo	Abnormal ECG (yes/no), CCS class, diastolic blood pressure, coronary artery dominance (R/L/B), familial IHD (yes/no), height, weight, ICD-device or PM (yes/no), ischemia test, LVEF, NYHA class, sex, smoking status, coronary pathology,
Diagnoses	322 different level-3 ICD-10 diagnosis codes.
Procedures	154 different NOMESCO procedure codes corresponding to various radiological examinations and surgical procedures
Biochemical	85 different lab tests with results categorized as below, within, or above the reference range

The different features were organized in five different categories each representing different domains. The clinical characteristics were divided into two subgroups where ClinicalOne contains the features used in the GRACE2.0 score

All features were defined using data recorded prior to the index date, except for creatinine, cardiac biomarkers for ischemic heart disease, and blood pressure where high missingness led us to allow measurements obtained after the index event in cases of high missingness (7-day threshold for cardiac biomarkers, and 21-day threshold for the others). The amount of missingness across the training and test set have been tabulated in Supplementary Table 1 and in Supplementary Data 2. In all our neural network models, missing values were left missing and explicitly encoded as such. All categorical features were one-hot encoded and an additional category for missing values was added if applicable [25]. All continuous features were Z-score normalized and missing values were encoded with a value of zero as is common practice [25].

During model development, we tested and optimized various feature specific hyperparameters. We tested including diagnosis codes (ICD-10) as either level 4, level 3, block, or chapter codes representing different granularity levels in the ICD-10 hierarchy [45]. With the hypothesis that codes and results assigned many years prior to time zero time might carry less information, we introduced three “shelf-life” hyperparameters and filtered out diagnosis codes, procedure codes, and biochemical tests assigned/taken more than n years prior to the index coronary angiography.

Additional details on laboratory test results

Laboratory test results and sex- and age adjusted (if applicable) reference ranges were originally stored in the regional laboratory information management systems “Labka” and “BCC” and in this study obtained through BTH [43]. Tests were either annotated in accordance with the Nomenclature, Properties and Units ontology (NPU) or various local coding systems [44]. Using the sex- and age adjusted reference ranges, results of the biochemical tests were discretized to the categories below, within, above, and if a given test had not been taken a category called missing was introduced. Discretization of biochemical tests using the adjusted reference ranges was a pragmatic alternative to normalizing the many different biochemical tests in a manner that adequately takes both intra- and interdepartmental variance into account.This variance might both be related to, for instance, differences in patient population and/or equipment and machinery. The approach has been described in detail by Muse and Brunak [26].

Polygenic risk scores of cardiometabolic health

Polygenic risk scores (PRSs) for patients with genotypes available through the CHB-CVD [19, 27] (37.4% of the cohort) were calculated using the LDpred2 framework, implemented in the R package bigsnpr (v1.5.2) with R version 3.5.052 [28]. PRSs were calculated based on GWAS summary statistics data from 19 traits relevant for cardiometabolic health, obtained from 17 GWAS meta-analyses. A list of meta-analyses and details on the PRS calculations is included in the Supplementary Information 1.

External validation in Iceland

For the external validation cohort, we identified Icelandic adults who had undergone coronary angiography at the only interventional cardiology center in Iceland, Landspitali–The National University Hospital in Reykjavík [29]. We obtained data collected prospectively between January 1, 2007, and December 31, 2017. Information on ICD-10 diagnoses and procedure codes were aggregated from the Landspitali, from registers kept by the Directorate of Health: The Register of Primary Health Contacts, the Register of Contacts with Medical Specialists in Private Practice and the Causes of Death Register, as well as at recruitment for deCODE studies. Biochemical assay measurements were obtained from the three largest clinical laboratories in Iceland, with measurements performed at: (i) Landspitali; (ii) The Laboratory in Mjodd, Reykjavik, Iceland; and (iii) Akureyri Hospital, the regional hospital in North Iceland.

Due to data availability, the PMHnet model was downscaled for the external validation (404 features, see Supplementary Data 1). This downscaled version was trained exclusively on Danish training data, and tested on the test set, to make direct comparisons. The features that were left out in the down-scaling were all procedure codes (n = 154) since they were encoded using another non-compatible coding scheme; five biochemical tests which were not used in the Icelandic system; and seven different clinical features for which data could not be obtained: “arrest”, “enzymes”, “abnormal-ekg”, “abnormal-qrs-st”, “coronary artery dominance”, “ischemia test”, and “NYHA-class”.

Discrete-time survival analysis with neural networks

To model time-to-event data, and allow for censoring, we used the discrete-time survival framework for neural networks described by Gensheimer and Narasimhan [30]. In this approach, follow-up time is discretized into a fixed number of intervals, and the model estimates a discrete hazard for each interval, i.e., the conditional probability of mortality in that time interval, given that the patient is still alive at the end of the preceding interval. Our implementation applied the PyTorch machine-learning framework using the original Keras-based version as a reference [30]. In PMHnet, we use 30 intervals separated in time such that event times in the training data are evenly distributed across all intervals. To obtain predictions between breakpoints in the discretization grid, we assumed that the probability density function was constant in each time interval, and thus interpolated using a piecewise linear function [31].

Machine learning model architecture and training

We used a feed-forward neural network with between one and three fully connected hidden layers with rectified linear units as the activation function and with a dropout layer after each hidden layer. The learning objective was all-cause mortality. We added dropout to each of the hidden layers to regularize the network and prevent over-fitting. The output layer has 30 outputs—one for each time bin—and is being fed through a sigmoid activation function to produce probabilities. The neural network architecture is sketched in Supplementary Fig. 5. The neural networks were trained using stochastic gradient descent, with a constant learning rate, to minimize the negative log-likelihood. The number of layers, neurons, learning-rate, and dropout rate for each layer were fine-tuned through hyperparameter optimization (see below). After hyperparameter optimization, the final model was trained on the entire training set using the fine-tuned parameters. The learning curves tracking train/test negative log-likelihood during training of the final model are included in Supplementary Fig. 6.

Hyperparameter optimization

For hyperparameter optimization we used the Optuna hyperparameter optimization package for python [32]. For all hyperparameter sweeps, we performed five-fold cross validation of the training set to get an unbiased estimate of model performance. For each of the different intermediate PMHnet models, we ran 2500 trials and sampled hyperparameters using the Tree-structured Parzen Estimator (TPE) algorithm as implemented in Optuna using 400 startup trials and otherwise all other arguments left on default settings [33]. To prune unpromising trials, we used the hyperband pruner, with a minimum resource of 20, and a reduction factor of 3. The hyperparameter search space and the best trial for the complete model, which were used for the final training, is included in Supplementary Table 3.

Model evaluation and validation

Using the hold-out test set and the external validation data from Iceland, performance of our machine learning models was evaluated through assessment of both model discrimination and calibration. We used time-dependent area under the receiver operating characteristic curve (tdAUC) as the main measure of discrimination, but also calculated the Brier score that can be used to assess both discrimination and calibration [34–36]. Calibration was also analyzed graphically by comparing the predicted risks with the estimated actual risks [34]. The "Score" function from the riskRegression R package was used to compute performance measures and compare models. For comparisons between two competing models, the Score function gives p-values that correspond to Wald tests on the standard errors obtained using an estimate of the influence functions, following Blanche et al. [36].

To benchmark the PMHnet model, we calculated the GRACE2.0 score for all patients in the hold-out test set. We extracted the javascript source code for the GRACE2.0 webtool using the developer tools in Google Chrome, and manually converted it to an R package to automate the process. The eight variables used in GRACE2.0 were available for 51.4% of the cohort. Since GRACE2.0 does not allow for missing features, we imputed missing variables using the missForest R package [37]. Imputed values were only used for calculating the GRACE2.0 score. As an additional comparison, we also trained a neural network-based equivalent using the PMHnet model limited to the GRACE2.0 features. This corresponds to model-2 in Fig. 4 that only uses the features from ClinicalOne as input.

Fig. 4.

Observed survival across 5-year predicted risk groups. a Estimated actual survival (Kaplan–Meier estimates) for patients in the test set manually stratified into five different risk-groups depending on the predicted survival at five-years by PMHnet. b Distribution of PMHnet 5-year predicted risk. Vertical lines show the cut-offs that are used to define the risk strata used in (a).

Explainability and effect of missing features

To investigate the impact of different features on model predictions and to provide model explanations, we calculated Shapley additive explanation (SHAP) values for all features and patients in the training set using the SHAP python package [38]. In the model explanations, a negative SHAP value for a given feature means that the feature pulls the prediction towards mortality and vice versa for positive SHAP values relative to the median prediction. The magnitude of the SHAP value is given as percentage points.

To assess the model’s resilience to missing data, missingness was introduced in the test data by replacing all values of a given feature with the median value of that feature in the training data. We compared the predictions with those obtained without missingness and quantified resiliency using change in tdAUC and the Brier score. This approach simulates a scenario where a feature is entirely unavailable when transferring the model to another setting. We used median imputation as a worst-case scenario for both continuous and categorical variables to evaluate how the absence of a feature would affect performance globally.

Statistical analysis

Categorical features are reported as counts (%) and continuous features as mean [95% CI], 95% CIs are obtained from standard deviations or through bootstrapping. tdAUCs and Brier scores were calculated using the riskRegression R-package [34]. The model comparisons were obtained from the same package. All statistical analyses and visualizations were performed using R version 4.1.

Results

Baseline characteristics

The derivation cohort of 39,746 Danish patients fulfilling criteria for inclusion was randomly subdivided into a training (n = 34,746) and a test set (n = 5000) (Table 1). At the time of the index coronary angiography, the patients mean age (95% CIs) was 66.0 years [65.7; 66.4] (67.3% males) in the training set and 66.2 years [66.0;66.3] (68.2% males) in the test set. Likewise, sex distributions and comorbidity burdens were similar. The distribution of coronary artery disease severity and other phenotypic characteristics was similar between the two groups. In general, no significant differences across baseline characteristics were observed between training and test sets (Table 1, Supplementary Data 1). The Kaplan–Meier estimate of five-year survival (all-cause) was 81.8% [81.4; 82.2] in the training set and 82.5% [81.3; 83.6] in the test set (Supplementary Fig. 1, Supplementary Table 2). The restricted mean follow-up time was 1,635 days (± 2.58) for patients in the training set and 1635 days (± 6.49) for the test set.

PMHnet evaluation and comparison with GRACE2.0

In the internal validation using the hold-out test set, the complete PMHnet model had a tdAUC of 0.88 [0.86; 0.90] at six months (case count, cc = 196), 0.88 [0.86; 0.90] at one year (cc = 261), 0.84 [0.82; 0.86] at three years (cc = 395) (Fig. 2), and 0.82 [0.80; 0.84] at five years (cc = 763). In comparison, the corresponding values for the GRACE2.0 score on the same dataset were 0.77 [0.74; 0.80] at six months, 0.77 [0.74; 0.80] at one year, and 0.73 [0.71; 0.75] at three years. Since GRACE2.0 features had to be imputed for 48.6% of the population to compute the score, we also evaluated the performance on the subset without missingness, which showed similar but slightly improved performance (Fig. 3, dashed line). The imputation was only performed to compute the GRACE2.0 score. An intermediate PMHnet model limited to the GRACE2.0 features, or “GRACE2.0 re-fitted”, that is trained on our data and allows for missingness had tdAUCs of 0.79 [0.76; 0.83] at six months, 0.78 [0.75; 0.81] at one year, and 0.76 [0.74; 0.78] at three years. GRACE2.0 does not provide predictions after three years and was therefore not evaluated beyond that time-point. D ifference in tdAUCs between PMHnet and either of the two GRACE2.0 models were statistically significant at each of the three prediction horizons (Table 3). We also assessed the model performance on females and male separately and observed no significant differences (Supplementary Table 4). As a supplemental qualitative test of model discrimination, we constructed five different risk strata using the 5-year predicted survival (defined as 90%, 75%, 50%, and 25%) and examined the observed survival (Fig. 4), which showed good separation between the five strata. Baseline characteristics of the five different risk strata are available in Supplementary Table 5. In terms of calibration, PMHnet was found to be well-calibrated as seen from Fig. 2b and the calculated Brier scores of 3.2% [2.8; 3.6] at six months, 4.1% [3.7; 4.5] at one year, 7.6% [7.1; 8.1] at three years, and 11.3% [10.5; 12.0] at five years. In comparison, GRACE2.0 had Brier scores of 3.7% [3.3; 4.1] at six months, 4.9% [4.4; 5.4] at one year, 9.9% [9.3; 10.5] at three years. Our neural network-based GRACE2.0 (GRACE2.0 re-fitted) improved calibration as evident from the calibration curve which was comparable to that of the full PMHnet model (Fig. 2b). The differences in three-year predicted risk between PMHnet and the two GRACE2.0 scores for all patients in the test set are shown in Supplementary Fig. 2.

Fig. 2.

Model performance of PMHnet and the GRACE2.0 score. a Time-dependent receiver operating characteristics (ROC) curves at three different prediction horizons for PMHnet, GRACE2.0 (re-fitted), and GRACE2.0 (conventional). Labels show the time-dependent area under the ROC curves (tdAUC). GRACE2.0 (re-fitted) is a model that uses the GRACE2.0 input features but uses the PMHnet architecture trained using the training data. b Calibration curves showing the relation between predicted risk and the estimated actual risk. Labels show the Brier score for each of the three models. Lower scores are associated with better calibration and discrimination of predictions.

Fig. 3.

Model discrimination with increasing number of feature categories. Time-dependent area under the curve (tdAUC) for various intermediate PMHnet models including the final one (model 6) at six months, one year, and three years after the index coronary angiography. Discrimination was evaluated using the hold-out test set. The colored boxes represent the different feature categories that were used as model input. Horizontal reference lines show the model discrimination of the GRACE2.0 score on the same data. The solid line is the tdAUC of GRACE2.0 on all patients and the dotted line is the tdAUC on the subset of patients where none of the GRACE2.0 input features were missing

Table 3.

Difference in discrimination between PMHnet and the GRACE2.0 score

	6 months		z 1 years		3 years
Comparison	ΔtdAUC (%)	p-value	ΔtdAUC (%)	p-value	ΔtdAUC (%)	p-value
PMHnet versus GRACE2.0
(conventional)	11.5 [9.0; 14.0]	1e−9	10.9 [8.6; 13.2]	6e−21	10.3 [8.3; 12.4]	1e−22
PMHnet versus GRACE2.0
(re-fitted)	8.9 [6.5; 11.2]	2e−13	8.9 [6.5; 11.2]	2e−18	7.6 [5.8; 9.4]	1e−6

For ΔtdAUC, we obtain 95% CIs (in brackets) and p-values from the Score function in the R package

External validation of PMHnet in Icelandic patients

For external validation the cohort of 8287 patients from Iceland was used. Generally, baseline characteristics were similar in the two populations. However, the prevalence of dyslipidaemia was higher in the Danish population. In contrast, P-creatinine levels were higher in the Icelandic population (Table 1). Due to data availability a modified, and slightly down-scaled version of PMHnet was used on the Icelandic data (404 features, see Supplementary Data 1). The tdAUCs were 0.87 [0.84;0.90] at six months, 0.84 [0.81;0.87] at one year, and 0.81 [0.79;0.83] at three years. In comparison, the performance of the down-scaled version on the Danish (internal, hold-out) test set was 0.87 [0.85;0.90] at six months, 0.87 [0.85;0.89] at one year, and 0.82 [0.80;0.85] at three years. While the performance of the Danish downscaled model was lower than it was when using all features, the performances of the two reduced models were quite similar, with sensitivities at all three prediction horizons of about 75%, if one aims at a specificity of 87,5%. As estimated by the Brier score, the Icelandic model maintained good calibration for the six months and one-year predictions, but contrasted with the Danish downscaled model found to be miscalibrated in the three-year predictions (Supplementary Fig. 3).

Influence of the different input feature categories on the PMHnet prediction

To assess the importance of the different input feature categories systematically, we trained five intermediate survival models using the PMHnet architecture (Fig. 3). For example, model-1 used only diagnosis codes as input features and model-2 used only the GRACE2.0 features. The other intermediate versions were trained using different combinations of the feature categories. Figure 3 shows the model discrimination at the three different prediction horizons for the six different models fitted using the PMHnet architecture. The performance of the intermediate model based on diagnosis codes only (model-1) was comparable the performance of the re-fitted GRACE2.0 model (model-2). Interestingly, combining diagnosis codes with the GRACE2.0 clinical features (Diagnoses + ClinicalOne) in model-3, increased tdAUC at three years, suggesting a synergistic effect. With the addition of the ClinicalTwo-data, the ΔtdAUC between model-3 and model-4 was 3.4e−2 [1.8e−2; 5.0e−2] at six months, 3.2e−2 [1.8e−2; 4.7e−2] at one year, and 1.3e−2 [0.3e−2; 2.3e−2] at three years. Similarly, adding Biochemical to the input features in model-5 associated with significant ΔtdAUCs of 2.1e−2 [0.2e−2; 3.6e−2] at six months, 1.4e−2 [0.2e−2; 2.7e−2] at one year, and a non-significant ΔtdAUC of 0.9e−2 [− 0.1e−2; 1.9e−2] at three years. The gain in discrimination from model-5 to the complete model-6 (PMHnet) by adding Procedures to the input features was only significant at the one-year prediction horizon.

Testing inclusion of polygenic risk scores in PMHnet

For 37.4% of the cohort, we had genotype information available and used that to calculate 19 different polygenic risk scores (PRS) that all related to different cardiometabolic traits. Limiting both the training set and the test set to only individuals with genotype data (37.4%), we tested adding the PRS scores to model-1 (Diagnoses), model-2 (Diagnoses + ClinicalOne), and model-6 (all features) and evaluated the model performance (Supplementary Fig. 4). Addition of the PRS did not significantly improve either model discrimination or calibration at any time-point (Supplementary Fig. 4a). On this basis the genotypes were not included in the final model.

Resiliency of PMHnet to missingness

To gauge the resiliency of our model, we tried replacing all values of a given feature with the median value of that feature in the training data. Across the 584 features, age was the only single feature, where the change in tdAUC changed significantly when artificially removed. In the case of model calibration, missingness of six individual features significantly affected the performance. These features were troponins, LVEF, age, smoking status, abnormal ECG, and coronary pathology. With these exceptions, these results are evidence that PMHnet is quite resilient to missing data, which again is consistent with the fact that deep-learning approaches have imputational power built into the design [39, 40].

Explainability analysis using SHAP

To quantify the impact of the different features on PMHnet predictions, i.e. explain the predicted outcome, we performed SHAP-analyses of the five-year model predictions. Across the five different feature categories, we found biochemical test results and diagnosis codes to be the most impactful, while the category ClinicalOne was least impactful (Fig. 5a). The most impactful diagnosis code was chronic obstructive pulmonary disease (ICD10: J44) and thus ranked higher than classical risk factors, such as type 2 diabetes. At the feature level, ‘age’, ‘number of affected vessels’ and ‘smoking’ were on-average the most predictive features. The top 25 features in terms of average model impact (average magnitude of SHAP-value), included ten features from Biochemical, nine from ClinicalTwo, three from ClinicalOne, two from Diagnoses, and one from Procedures (Fig. 5a). Some important features, like ‘body weight’ and ‘body height’ may confer value through interactions with other biochemical and clinical features that often need to be gauged against anthropometrics. In this way they increase the predictive value of other features, hence their importance, as reflected by the SHAP analysis. This does not mean that they independently have high predictive value. To further examine the impact on model prediction for the two most impactful features, ‘number of affected vessels’ and ‘age’, we constructed SHAP dependence plots (Fig. 5b) [41]. For both features, we noted that identical feature values are not always impacted the survival to the same extent. This vertical dispersion in both plots represent interaction effects with the other included features [41]. Although our SHAP analyses revealed that several features have lower impact relative to the most impactful features, the aggregated sum of the many low-impact features (560 lowest) outweighs many of the well-known risk-factors (25 highest). For this reason, we did not attempt to limit the number of included features. Extending the analysis of feature-level model impact to the rest of the top 25 features, we constructed a summary plot of the SHAP-values that shows the distribution of model impacts across feature values (Fig. 6). To further interrogate the differential feature importance, for example the contribution from the age feature, we repeated training of the entire model for all four prediction horizons wihtout age as input feature. The performance drop is slightly higher for the 5-year prediction horizon, while they are highly similar for the other three prediction horizons. This is expected, as the method to a high degree will be able to infer the age from all the other features, for example the disease history. The opposite is of course not correct; the age alone is not predictive. This is consistent with the already included SHAP analysis that quantifies the relevance of the different features (Supplementary Table 6).

Fig. 5.

Overview of feature importance. Summary of the results from SHAP analysis on the model predictions at five years on patients in the Danish test set. a Left: Relative feature importance aggregated across the five different feature categories. Biochemical test results and diagnoses were found to affect the model prediction the most. Right: Relative feature importance for all singular features included in the model. Features arranged according to SHAP-values and labels are included for the top 25-most impactful features. Color of features correspond to the feature category in which they belong. b Relationship between age and SHAP-value with each point showing the SHAP-value for age for a patient in the test set, and relationship between coronary pathology (e.g. vessel status) and impact on model prediction. SHAP: SHapley Additive exPlanations.

Fig. 6.

SHAP summary plot for top 25 most impactful features. Left: average magnitude of model impact. Y-axis labels specify the feature name and inside parentheses is shown the unit or the factor levels, for continuous and categorial features, respectively. Right: Distribution of feature impacts across the test set. For continuous features, the colours correspond to the feature value ranging from blue (smallest) to red (largest). For categorical features, the factor levels are coloured blue or red. Grey indicates missingness. SHAP: SHapley Additive exPlanations.

Patient-level feature importance

For illustrative purposes we generated individual explanations for three patient cases in the test set and identified the SHAP values for the nine most impactful features (Fig. 7). The patients were randomly selected from the subsets of patients with a prediction in the intervals (0.25; 0.5], (0.5, 0.75], and (0.75; 1] and their individual characteristics were slightly altered to make them non-person identifiable. This figure illustrates that features may impact the prediction differently in different representative cases. Overall, one can observe that the intermediate- and low-risk patient (middle and right panel, respectively), derive contributions from lower number of features compared to high-risk patient (left panel). Further, age is only among the top nine predictive features for the intermediate- and low-risk patient, respectively. In contrast, for the high-risk patient, 50% of the top nine predictive features are comorbidities (diagnosis codes). For the low-risk patient, smoking shows up as a feature that likely contributes to overruling the other features that tend to indicate good health. One also must note that the features may reflect disease management, as the case with high pO2 in the high-risk patient is indicative of exogenous oxygen supplement. Taken together, the representative cases illustrate how the algorithm compute risks differently (Fig. 7).

Fig. 7.

Patient-level model explanations with SHAP. Model predictions for three different representative patients with a predicted 5-year survival of 42%, 73%, and 98% with SHAP-value showing the estimated impact on model prediction. The patient data have been slightly adjusted to make them non-identifiable. Blue represents features that contribute positively to the prediction. Red represents features that contribute negatively to the prediction. The dashed line is the median prediction from the training set patient and the solid line is the prediction for each of the example patients. For patient 1, with the worst 5-year prognosis, age was not among the nine most impactful features. Instead, the explainability algorithm highlights diagnosis codes and biochemical values. For patient 2 with a 5-year predicted risk of 73%, the most impactful feature was age (78 years), which pulled the prediction towards mortality. The impact of age was however largely cancelled by the rest of the features, which constitute the aggregated sum of all the features not among the nine most predictive ones. For patient 3, the feature age was again highlighted as the most impactful, but in this case, it impacted the prognosis positively. Apart for a history of cigarette smoking, none of the highlighted features had negative impact on the prognosis. SHAP: SHapley Additive exPlanations.

Discussion

In this study, we developed and validated PMHnet, a feature-rich neural network-based time-to-event model for prediction of all-cause mortality in patients with ischemic heart disease. We developed a new prediction model to support and guide choice of management. This model, PMHnet, operates with a prediction horizon of five years and provides individualised, explainable, and time-dependent probabilistic estimates for survival after patients have received a coronary angiography-confirmed diagnosis of coronary artery disease. We trained, evaluated, and externally validated the model using real-world hospital data from thirty-four thousand Danish patients, five thousand Danish patients, and eight thousand Icelandic patients, respectively. Both the Danish and Icelandic cohorts are population-based and contain a distribution of high- and low-risk patients. While the pooling of subgroups in ischemic heart disease may at first appear to be counterintuitive, we argue that value of a risk scoring scheme that can be used in the catheter lab irrespective of the underlying phenotype is a strength. This is in analogy to pan-cancer predictors, which often seem to benefit from mixtures of phenotypes and the additional patients then included as compared to focusing on a very narrow condition.

Both the Danish and Icelandic healthcare systems are of the one-payer type, admitting from the general population. We found that PMHnet had excellent discrimination with tdAUCs ranging from 0.88 at six months to 0.82 at five years and the model was well-calibrated with predicted probabilities accurately reflecting the observed proportions, also in different risk strata. The external validation, using data from an entirely different country, showed comparable results on a downscaled model, where input features were fully compatible. This raises the question of the added value of the full model over downscaled models, but also that the model seems to be quite robust to removal of the least predictive features.

While the GRACE2.0 score is widely considered the clinical gold-standard risk-stratification tool in patients with a diagnosis of ischemic heart disease [42], it was not developed exactly for the cohort definition applied in this work. However, the GRACE2.0 score was developed on and intended to be used for patients with acute coronary syndrome, which is not identical to the cohort definition used in this study. Be that as it may, PMHnet had statistically significant superior discrimination and calibration across all evaluated prediction timepoints compared to the GRACE2.0 score. The GRACE score has been developed using a derivation cohort of patients with STEMI, n-STEMI, and unstable angina with time of initial admission as time-zero [6]. In contrast, our model uses time of invasive coronary angiography as its baseline and applies to all patients with suspected or diagnosed ischemic heart disease subjected to this procedure. The validity of GRACE in such a patient population has not explicitly been established. To provide a more direct comparison of the two algorithms, we used the eight GRACE features and created a ‘re-fitted’ GRACE2.0 using our training data. This re-fitted score had the same model discrimination and better model calibration than the original GRACE ‘web-tool’ version (Fig. 2), but still had inferior prediction compared to PMHnet. This agrees with current literature in the domain, where ML-based prognostication has repeatedly been shown to offer better performance than existing alternatives based on classical statistical methods [11–14, 43, 44].

Our findings suggests that inclusion of additional features is necessary for obtaining more accurate models and that a transition towards feature-rich models that utilize more of the available data can be an advantage. Again, this aligns with established literature where feature-rich ML-based approaches have shown better prediction performance than traditional clinical scores [13, 45, 46]. In our study, the 584 distinct features used in our final model are neither bespoke nor specifically collected for this decision-support application, and instead represent clinical information gathered during routine work-up, management, and treatment of patients. Although our larger models generally perform best, the number of included feature sets can most likely be scaled back without large losses in performance. However, since the main objective of our study is to develop a multi-feature prediction tool for the Danish health care system where these features are readily available, we have not explored such minimization further.

To aid the clinical interpretation of predictions from a model of this complexity, explainability analysis, such as SHAP-values, can be used to quantify the impact of individual-level features and help explain why a given prediction was made, which is likely paramount for any potential clinical adoption [47]. Importantly, a feature may have predictive value as a proxy for something that is causal. This could be an examination of an organ not related to ischemic heart disease per se, but an event that is indicative of the general health of the patient rather than something causal. From the SHAP analysis, we found that the aggregated impact of many less predictive features outweighs the impact of those few being highly predictive. Given that age was identified as the most influential single feature, it follows that the risk strata shown in Fig. 4 would differ in age distribution. However, the substantial vertical dispersion in SHAP values at any given age (Fig. 5B) indicates that predicted risk is not reducible to age alone, which is also consistent with the performance of the model trained on data without age as input feature (Supplementary Table 5). Therefore, we argue that it is an inherent strength of the study that no strict feature selection criteria were applied. Neural networks are by design capable of handling correlated and potentially redundant data, so a more lenient feature inclusion policy is likely the best option. However, using models that rely on several hundred features means that the current practice of manually entering data into a webtool becomes very impractical [48]. Instead, novel risk-prediction models such as ours need to be integrated in EHR systems for automatic data transfer. The clinical implementation of PMHnet is beyond the scope of present study [49].

As a secondary analysis in this study, we experimented adding a panel of PRSs to the input features in the final PMHnet model and assessed their impact on model performance. We included 19 different genetic risk scores all broadly related to cardiometabolic health which covered traits such as ‘blood pressure’, ‘total cholesterol’, ‘heart failure’, and ‘acute myocardial infarction’. While the two latter represent risk scores developed for primary prevention, or disease development, our model was intended for use in a secondary prevention-setting with prediction of risk in those who already have a diagnosis of ischemic heart disease. It is known that a coronary artery disease genetic risk score associates with events in both primary and secondary prevention populations, however, these risk scores predict prevalent disease, not mortality [50]. Whether a primary prevention score is useful in a secondary prevention context is unclear upfront, but as parts of the underlying disease process are shared between the two, we hypothesized that such scores could potentially be useful. Focusing the analysis only on the subset of patients for which we could obtain genotypes (n = 13,449 or 37.4%), we found no significant difference in model discrimination after adding the polygenic scores. Our interpretation is that a “realized” life-course disease trajectory is more informative than the germline risk calculated from the genotype. Moreover, the disease trajectory also holds information on life-course exposures that the PRSs will not reflect systematically. The version of PMHnet trained on the disease history only, with and without genetics, indicates that there is little additional gain from the genetic risks scores. As we did not have genetic data for the full cohort, we cannot exclude that our interpretation is affected by this aspect.

From a methodological perspective, our work has several strengths that are worth highlighting. In contrast to most of the established literature in the domain, we here made use of a neural network architecture that enables modelling of time-to-event data. Instead of treating t-year mortality as a binary classification problem [13–15, 51, 52], time-to-event modelling enables the use of censored observations with incomplete follow-up [53] and can distinguish between ‘died after a week’ and ‘died after 10 months’. The use of such ML-based approaches remains less explored, with few exceptions [12, 54–57]. Additionally, our model was developed on a large real-world dataset and was furthermore externally validated on, likewise, routine health care data from a different country. Most comparable models in the literature were either not or sub-optimally externally validated [11, 12, 43, 44], and rarely using data from a different country [13, 14].

In terms of limitations, although the features we have used represent data collected from a typical clinical workflow and therefore should be generally applicable, inter-regional and inter-national differences in clinical practice may affect what data is available and when. This is for instance exemplified by differences in diagnostic work-up, or timely access to coronary angiography, but may also relate to differences in access to previous medical data. Such aspects could affect the generalizability of our included features, but with internationally accepted treatment guidelines the differences should be minimal. For the purposes of this study, which aims to develop and implement a prediction tool specifically for the Danish healthcare setting, the immediate concern of broad scale generalizability is less pressing. Our primary focus is on creating a model that performs well within our target environment, with the intention of exploring wider applicability and cross-population validation in subsequent research. Practical solutions include reducing the complexity of models to better match the intersection of features available across countries/regions and/or re-fitting and possibly retraining the model were it to be deployed in a new setting. In our external validation we used data from Iceland, which in an international perspective is very similar to Denmark when comparing healthcare systems [58, 59]. We downscaled the model slightly to account for data availability, but re-training on Icelandic data was not deemed necessary. Using the Icelandic data, we found the model on average to be well-calibrated, but from visual inspection of the calibration curve found evidence of miscalibration as the model was found to overestimate the risk for some patients which would suggest that further adjustment of the model is needed were it to be deployed in Iceland. However, since miscalibration does not affect the accurate risk-stratification of patients, we did not pursue that issue further in this study. For future studies, we note that techniques such as Platt-scaling or isotonic regression could be used to remedy calibration-issues [60]. Another limitation worth highlighting is our choice of discretizing biochemical results into categorical values, which might lead to a loss of otherwise valuable information. While continuous features would be preferable, our decision was driven by practical considerations. Harmonizing lab data across time periods and accounting for changes in machinery and reagents is complex and time-consuming. As a pragmatic solution, we used the reference ranges used by the hospital biochemical departments, which have already been adjusted for things like sex and age. This approach, which has been thoroughly described elsewhere, offers effective harmonization and reflects real-world clinical practice, where test results are usually interpreted as either normal or abnormal [26].

In routine cardiology, a multitude of diagnostic, prognostic, and treatment-related scores are applied. However, all the presently applied scores are typically based on 5–10 features, yet they often come with a specific set of recommendations. The present findings indicate a significantly added value of applying far more features in combination with a neural network-based prediction tool. Future precision management and treatment in cardiology may benefit from transitioning toward such feature-rich ML prognostication scores instead of currently used alternatives, but prospective clinical studies are needed to ascertain the real-life clinical utility. As described by Shah et al., such real-life clinical utility would also include a specific set of treatment recommendations, as well as other outcomes than all-cause mortality, while also considering the care environment, which is what matters most to patients [61]. Ideally, outcomes should match the intervention of choice or take plausible alternatives into account, as it is not necessarily the model with the highest sensitivity and specificity that will have the highest utility in clinical practice [61]. In its current state, the PMHnet algorithm is being tested in a randomized controlled trial (registered at clinicaltrials.gov: NCT06033014) that assesses if knowledge of the prognosis per se decreases mortality and other outcomes, such as over- and undertreatment. Striking the correct balance between over- and undertreatment can contribute to the advancement of precision medicine, and here a well-calibrated and highly discriminative risk-prediction model can serve as an important tool.

Author contributions (CRediT)

PCH: conceptualization, methodology, software, formal analysis, investigation, visualization, writing—original draft, writing—review & editing. ADH: conceptualization, formal analysis, data curation, writing—original draft, writing—review & editing. DW: methodology, writing—original draft, writing—review & editing. TR: software, formal analysis KB: methodology, supervision, writing—original draft, writing—review & editing. VT: validation, formal analysis. CHJ: formal analysis, writing—review & editing. AHC: writing—review & editing. LT: writing—review & editing. THN: writing—review & editing. AHS: writing—review & editing. KKI: writing—review & editing. FP: data curation, writing—review & editing. DEH: data curation. OBP: data curation. SRO: data curation, funding acquisition. HU: data curation, funding acquisition. MNS: writing—review & editing. IMG: writing—review & editing. TG: data curation. DFG: validation. AH: writing—review & editing. KH: data curation. LVK: writing—review & editing. HH: validation, writing—review & editing. KS: validation, resources, funding acquisition. SB: conceptualization, methodology, resources, funding acquisition, project administration, supervision, writing—original draft, writing—review & editing. HB: conceptualization, funding acquisition, project administration, supervision, writing—original draft, writing—review & editing.

Funding

Open access funding provided by Copenhagen University. The work was funded by Novo Nordisk Foundation (NNF14CC0001 and NNF17OC0027594); NordForsk (PM Heart, #90580); and the Innovation Foundation Denmark (BigTempHealth, 5153-00002B).

Data availability

Due to national and EU regulations, we cannot publicly share data used for model development and validation. Research groups with access to secure and dedicated computing environments can request access to the source data registries via application to the Danish Health Data Authority.

Code availability

Python and R code used for model training, hyperparameter optimization, model evaluation, etc. can be accessed at GitHub (https://github.com/peterchristofferholm/pmhnet-v1). This repository also includes a Dockerfile which specifies the software requirements and library versions used throughout our study.

Declarations

Ethics approval and consent to participate

The study was approved by The National Ethics Committee (1708829, ‘Genetics of CVD’—a genome-wide association study on repository samples from CHB), The Danish Data ProtectionAgency (ref: 514-0255/18-3000, 514-0254/18-3000, SUND-2016-50), The Danish Health DataAuthority (ref: FSEID-00003724 and FSEID-00003092), and The Danish Patient Safety Authority (3-3013-1731/1/). The National Danish Ethics Committee have waived the need for informed consent due to the societal value of the study. Danish personal identifiers were pseudonymised prior to any analysis.

The validation part of the study was approved by the Data Protection Authority of Iceland and the National Bioethics Committee of Iceland (VSN-15-114). Icelandic participants that donated biological samples provided informed consent. Personal identities of the participants were encrypted with a third-party system provided by the Data Protection Authority of Iceland.

Study design, methods, and results were reported in agreement with the TRIPOD statement [62, 63] and following the STROBE recommendations [64]. A filled-out TRIPOD checklist is included in the Supplementary Information document.

Competing interests

Lars V. Køber reports speakers honorarium from AstraZeneca, Boehringer, Novartis and Novo Nordisk. Søren Brunak reports ownerships in Hoba Therapeutics, Novo Nordisk, Eli Lilly & Co., andLundbeck, and has received lecture fees from MDS, Bayer, LEO Pharma, Roche and Astellas. Henning Bundgaard reports ownership in Novo Nordisk and has received lecture fees from Amgen, BMS, MSD and Sanofi. The following co-authors are or were employed by deCODE genetics/Amgen, Inc: Vinicius Tragante, Daníel F. Guðbjartsson, Anna Helgadottir, Hilma Holm, and Kari Stefansson. The rest of the authors declare no competing interests.