Table 3.
Model performance parameters
| AUC | Sensitivity | Specificity | Precision score | DeLong* | |
|---|---|---|---|---|---|
| Centre 1 (training cohort) all ejection fraction | |||||
| Random forest | 0.99 (0.98–0.99) | 93.3% | 97.3% | 93.8% | N/A |
| Logistic regression | 0.93 (0.91–0.95) | 81.2% | 95.0% | 82.2% | <0.001 |
| ADHERE | 0.68 (0.65–0.71) | 75.9% | 49.4% | 15.0% | <0.001 |
| GWTG‐HF | 0.69 (0.66–0.72) | 64.5% | 63.0% | 14.6% | <0.001 |
| Centre 1 (training cohort) heart failure with reduced ejection fraction (ejection fraction ≤40%) | |||||
| Random forest | 0.99 (0.98–1.00) | 94.1% | 96.8% | 95.1% | N/A |
| Logistic regression | 0.90 (0.86–0.94) | 83.9% | 86.9% | 71.1% | 3.11E‐03 |
| ADHERE | 0.68 (0.63–0.72) | 59.3% | 66.8% | 18.1% | <0.001 |
| GWTG‐HF | 0.64 (0.59–0.69) | 52.5% | 68.5% | 15.8% | <0.001 |
| Centre 1 (training cohort) heart failure with mildly reduced ejection fraction (ejection fraction 41–49%) | |||||
| Random forest | 1.00 (1.00–1.00) | 100.0% | 99.6% | 99.5% | N/A |
| Logistic regression | 0.73 (0.59–0.87) | 79.0% | 67.6% | 31.6% | 2.33E‐02 |
| ADHERE | 0.67 (0.54–0.80) | 79.0% | 49.6% | 18.7% | 1.59E‐03 |
| GWTG‐HF | 0.66 (0.53–0.80) | 68.4% | 62.3% | 15.6% | 2.59E‐03 |
| Centre 1 (training cohort) heart failure with preserved ejection fraction (ejection fraction ≥50%) | |||||
| Random forest | 0.98 (0.97–1.00) | 94.5% | 93.9% | 92.5% | N/A |
| Logistic regression | 0.91 (0.88–0.95) | 77.9% | 94.7% | 77.1% | 5.93E‐03 |
| ADHERE | 0.68 (0.64–0.72) | 60.0% | 66.7% | 13.2% | <0.001 |
| GWTG‐HF | 0.57 (0.53–0.62) | 44.1% | 71.1% | 9.0% | <0.001 |
| Centre 2 (validation cohort) all ejection fraction | |||||
| Random forest | 0.96 (0.95–0.97) | 87.3% | 90.6% | 79.9% | N/A |
| Logistic regression | 0.87 (0.85–0.89) | 75.9% | 86.9% | 63.4% | <0.001 |
| ADHERE | 0.70 (0.67–0.72) | 69.0% | 61.7% | 11.4% | <0.001 |
| GWTG‐HF | 0.69 (0.67–0.72) | 66.8% | 62.9% | 11.1% | <0.001 |
| Centre 2 (validation cohort) heart failure with reduced ejection fraction (ejection fraction ≤40%) | |||||
| Random forest | 0.96 (0.94–0.98) | 88.2% | 91.0% | 81.6% | N/A |
| Logistic regression | 0.85 (0.81–0.89) | 64.6% | 90.1% | 54.8% | <0.001 |
| ADHERE | 0.69 (0.65–0.74) | 70.8% | 60.9% | 15.8% | <0.001 |
| GWTG‐HF | 0.55 (0.50–0.60) | 41.0% | 68.5% | 8.3% | <0.001 |
| Centre 2 (validation cohort) heart failure with mildly reduced ejection fraction (ejection fraction 41–49%) | |||||
| Random forest | 0.95 (0.88–1.00) | 90.0% | 95.3% | 83.1% | N/A |
| Logistic regression | 0.80 (0.69–0.90) | 75.0% | 77.9% | 20.2% | 1.72E‐01 |
| ADHERE | 0.70 (0.58–0.81) | 70.0% | 72.6% | 9.8% | 2.17E‐02 |
| GWTG‐HF | 0.71 (0.60–0.83) | 75.0% | 67.3% | 10.7% | 3.53E‐02 |
| Centre 2 (validation cohort) heart failure with preserved ejection fraction (ejection fraction ≥50%) | |||||
| Random forest | 0.95 (0.93–0.97) | 87.4% | 89.5% | 76.9% | N/A |
| Logistic regression | 0.84 (0.81–0.88) | 73.0% | 84.5% | 54.0% | <0.001 |
| ADHERE | 0.70 (0.66–0.73) | 66.5% | 62.8% | 9.9% | <0.001 |
| GWTG‐HF | 0.59 (0.55–0.63) | 35.2% | 79.7% | 7.7% | <0.001 |
*
P value, compared with random forest.
Abbreviations: ADHERE, Acute Decompensated Heart Failure National Registry; AUC, area under the curve; GWTG‐HF, Get With the Guidelines Heart Failure.