Table 2.
Model discrimination and calibration
| Study | Model | Setting | Predictors; n | Cohort | Discrimination | Type calibration | Calibration |
| Moretti et al57 | EuroHeart PCI score | ACS | 16 | External | 0.59 (0.48–0.71) | NA | |
| Asche et al46 | NR | AMI | 19 | Development; random split | 0.74; NR | NA | |
| Cediel et al58 | TARRACO risk score | AMI type 2; ischaemia | 7 | Development (30d) | 0.71 (0.61–0.82) | NA | |
| AMI type 2; ischaemia | 7 | Development (180d) | 0.71 (0.64–0.78) | NA | |||
| Burke et al35 | HOSPITAL score | AMI | 7 | External | 0.66 (0.61–0.71) | HLT | p=0.49 |
| Chotechuang et al36 | GRACE | AMI | 9 | External (30d) | 0.77 (0.65–0.88) | NA | |
| GRACE | AMI | 9 | External (180d) | 0.63 (0.49–0.77) | NA | ||
| Hilbert et al59 | AMI decision tree | AMI | 44 | Development; External | 0.65 (0.64–0.66) 0.61 (0.61–0.62) |
NA | |
| Dodson et al18 | SILVER-AMI 30-day readmission calculator | AMI | 10 | Development; random split | 0.65; 0.63 | HLT | p>0.05; p=0.05 |
| Kini et al60 | NR | AMI | 12 | Development; random split | NR; 0.66 | Slope; in large; plot | 0.973 (p=0.330); −0.038 (p=0.221) |
| Nguyen et al19 | AMI READMITS score | AMI | 7 | Development; random split | 0.75 (0.70–0.80) 0.73 (0.71–0.74) |
Plot; plot | |
| Full-stay AMI model | AMI | 10 | Development; random split | 0.78 (0.74–0.83) 0.75 (0.74–0.76) |
Plot | ||
| CMS AMI administrative model | AMI | 32 | External | 0.74 (0.69–0.74) | Plot | ||
| Krumholz et al20 | CMS AMI administrative model | AMI | 32 | Development; external; random split | 0.63; 0.63; 0.62 | In large; slope | |
| CMS AMI medical model | AMI | 45 | Development; random split | 0.58; 0.59 | NA | 0, 1/0.015; 0.997/0.015; 0.983 | |
| Rana et al33 | Elixhauser index | AMI | 30 | External | 0.53 (0.42–0.65) | NA | |
| HOSPITAL core | AMI | 7 | External | 0.60 (0.47–0.73) | NA | ||
| Atzema et al47 | AFTER Part 2 scoring system | Arrhythmia; AF | 12 | Development | 0.69; NR | NA | |
| Lahewala et al40 | CHADS2 | Arrhythmia; AF | 5 | External (30d) | 0.64 | NA | |
| CHADS2 | Arrhythmia; AF | 5 | External (90d) | 0.63 | NA | ||
| CHA2DS-VASc | Arrhythmia; AF | 9 | External (30d) | 0.65 | NA | ||
| CHA2DS-VASc | Arrhythmia; AF | 9 | External (90d) | 0.63 | NA | ||
| Benuzillo et al61 | CRSS | CABG | 5 | Development; bootstrapping | 0.63; 0.63 | HLT | 7.13 (p=0.52); 9.31 (p=0.32) |
| Deo et al62 | 30-days CABG readmission calculator | CABG | 20 | Development | 0.65 | NA | |
| Engoren et al55 | NR | CABG | 6 | Development; random split | 0.68 (0.64–0.72) 0.68 (0.64–0.68) | NA | |
| Lancey et al63 | NR | CABG | 8 | Development; random split | 0.64; 0.57 | NA | |
| Rosenblum et al41 | The STS PROM score | CABG | 40 | External | 0.59 (0.57–0.60) | NA | |
| Zitser-Gurevich et al64 | NR | CABG | 17 | Development; external (30d) | 0.63; 0.66/0.63 | HLT | 7.91 (p=0.44) |
| NR | CABG | 13 | Development (100d) | 0.65 | HLT | 6.76 (p=0.56) | |
| Zywot et al42 | CABG risk scale | CABG | 27 | Development; external | NR; 0.70 | Plot | |
| Ahmad et al21 | CMS HF administrative model | HF | 37 | External | 0.66 (0.57–0.76) | HLT | p=0.19 |
| Amarasingham et al22 | ADHERE | HF | 3 | External | 0.56 (0.54–0.59) | NA | |
| CMS HF administrative model | HF | 37 | External | 0.66 (0.63–0.68) | NA | ||
| Tabak mortality score | HF | 18 | External | 0.61 (0.59–0.64) | NA | ||
| Au et al23 | Administrative claims model, HF 30-day mortality | HF | 17 | External | 0.58 (0.58–0.59) | NA | |
| Charlson Comorbidity Score | HF | 32 | External | 0.55 (0.55–0 56) | NA | ||
| CMS HF administrative model | HF | 37 | External | 0.59 (0.59–0.60) | NA | ||
| LACE | HF | 18 | External | 0.58 (0.58–0.59) | NA | ||
| Bardhan et al65 | NR | HF | 30 | Development | 0.56 | NA | |
| Betihavas et al66 | NR | HF | 7 | Development; bootstrapping | NR; 0.80 | NA | |
| Burke et al35 | HOSPITAL score | HF | 7 | External | 0.67 (0.65–0.70) | HLT | p=0.10 |
| Cox et al24 | CMS HF administrative model | HF | 37 | External | 0.61 | NA | |
| CMS HF medical model | HF | 20 | External | 0.60 | NA | ||
| Delgado et al67 | 15-day CV readmission risk score | HF | 5 | Development; bootstrapping | 0.65; 0.63 | Plot | |
| 30-day CV readmission risk score | HF | 11 | Development; bootstrapping | 0.66; 0.64 | Plot | ||
| Formiga et al30 | CMS HF medical model | HF | 19 | External (30d) | 0.65 (0.57–0.72) | NA | |
| CMS HF medical model | HF | 19 | External (90d) | 0.62 (0.56–0.68) | NA | ||
| Frizzell et al25 | CMS HF administrative model | HF | 37 | External | 0.60 | NA | |
| Hammill et al26 | CMS HF administrative model | HF | 37 | External | 0.59 | Plot | |
| Hilbert et al59 | HF decision tree | HF | 44 | Development; External | 0.59 (0.58–0.60) 0.58 (0.58–0.59) |
NA | |
| Hummel et al31 | CMS HF medical model | HF | 28 | External | 0.61 | NA | |
| Huynh et al48 | NR | HF | 12 | Development; external (30d) | 0.82 (0.76–0.87) 0.73 (0.69–0.77) |
NA | |
| NR | HF | 12 | Development; external (90d) | NR; 0.65 | NA | ||
| Ibrahim et al34 | HOSPITAL score | HfpEF | 7 | External | 0.60 (0.55–0.64) | NA | |
| LACE | HfpEF | 18 | External | 0.55 (0.50–0.60) | NA | ||
| LACE+ index | HfpEF | 24 | External | 0.57 (0.52–0.62) | NA | ||
| Keenan et al27 | CMS HF administrative model | HF | 37 | Development; external; random split | 0.60; 0.60; 0.61 | In large; slope | 0, 1/0.02; 1.01/ 0.09; 1.05 |
| CMS HF medical model | HF | 30 | Development; random split | 0.58; 0.61 | In large; slope | 0, 1/0, 1 | |
| Kitamura et al53 | FIM | HF | 13 | External | 0.78 | NA | |
| Leong et al68 | 30-day HF readmission risk score | HF | 7 | Development; random split | 0.76; 0.76 | NA | |
| Li et al49 | NR | HF | 10 | Development; random split | 0.63 (0.62–0.63) 0.63 (0.62–0.63) |
HLT; plot | 0.15 (p>0.005) |
| Lim et al69 | NR | HF | 13 | Development | 0.68 (car); 0.62 (all) | HLT | 27.5 (p=0.001) (car) 8.0 (p=0.429) (all) |
| Reed et al28 | AH model | HF | 14 | Development; random split | 0.86 (0.85–0.86) 0.85 (0.84–0.86) | NA | |
| CMS HF administrative model | HF | 37 | Random split | 0.55 (0.54–0.56) 0.55 (0.54–0.57) |
NA | ||
| Hasan | HF | 9 | Random split | 0.80 (0.79–0.81) 0.80 (0.80–0.82) |
NA | ||
| LACE | HF | 18 | Random split | 0.75 (0.74–0.81) 0.74 (0.73–0.76) |
NA | ||
| Reed et al (continued)28 | PARR-30 | HF | 10 | Random split | 0.82 (0.81–0.83) 0.81 (0.80–0.82) |
NA | |
| Salah et al70 | ELAN-HF Score | HF | 10 | Development | 0.60 (0.56–0.64) | NA | |
| Sudhakar et al32 | CMS HF medical model | HF | 20 | External | 0.61 (0.57–0.64) ≥65 y, 0.59 (0.53–0.64) Random patient-level, 0.58 (0.50–0.65) |
NA | |
| Tan et al71 | NR | HF | 3 | Random split | 0.73 | HLT; plot | p=0.62 |
| Wang et al72 | NR | HF | 12 | Development | 0.65 | NA | |
| Wang et al38 | LACE | HF | 18 | External | 0.56 (0.48–0.64) | NA | |
| Yazdan-Ashoori et al29 | CMS HF administrative model | HF | 37 | External | 0.61 (0.55–0.67) | NA | |
| LACE | HF | 18 | External | 0.59 (0.52–0.65) | HLT | p=0.73 | |
| Disdier Moulder et al73 | NR | HF; ACS; NR | 4 | Development (30d) | 0.68 | NA | |
| NR | HF; ACS; NR | 5 | Development (180d) | 0.69 | NA | ||
| Raposeiras-Roubín et al37 | GRACE | HF; ACS | 9 | External | 0.74 (0.73–0.80) | HLT | p=0.14 |
| Minges et al74 | NR | HF; PCI | 35 | Development; random split | 0.67; 0.66 | NA | |
| Pack et al75 | NR | HVD | 28 | Development; random split | 0.67 (full dev)/ 0.65 (nomogram); 0.67 (full val) |
Harrell’s E; O, E; Harrell’s E; plot | 0.1%; 1.9%; 1.6% |
| Oliver-McNeil et al76 | ICD readmission-risk score | ICD | 4 | Update; External | 0.69 (0.58–0.79) | HLT; plot | 3.44 (p=0.49) |
| Wasfy et al52 | Pre-PCI model | NR | 23 | Development; random split | 0.68; 0.67 | HLT; plot | p=0.59 |
| Barnett et al77 | NR validation | Surgical | 15 | External | 0.59 | NA | |
| NR update | Surgical | 18 | Update | 0.60 (0.59–0.62) | NA | ||
| Brown et al43 | STS augmented clinical model | Surgical | 27 | Update (bootstrap); random split; external (bootstrap) | 0.66 (0.61–0.72); 0.56; 0.47 (0.42–0.53) |
HLT | p=1.0 |
| STS 30-day readmission model | Surgical | 21 | Update (bootstrap); random split; external (bootstrap) | 0.66 (0.62–0.71), 0.58, 0.47 (0.41–0.52) |
HLT | p=0.492 | |
| Espinoza et al78 | 30-day readmission score after cardiac surgery | Surgical | 5 | Development; random split | 0.66 (0.63–0.70) 0.64 (0.61–0.67) |
NA | |
| Ferraris et al54 | READMIT | Surgical | 9 | Development | 0.70 | HLT | 5.966 (p=0.651) |
| Kilic et al79 | NR | Surgical | 15 | Development; random split | NR; 0.64 | HLT; plot | p=0.45; p=0.57 |
| Stuebe et al80 | NR | Surgical | 7 | Development | 0.63 | NA | |
| Tam et al44 | NR | Surgical | 29 | Development; bootstrapping | 0.63; 0.65 | Plot | |
| Khera et al45 | TAVR 30-Day readmission risk model | TAVR | 11 | Development; random split; external | NR; 0.63; 0.69 | HLT; RMSE; RMSE; plot | p=0.33; 0.978; 0.928 |
| Sanchez et al50 | NR | TAVR | 10 | Development; random split | 0.61; 0.60 | HLT | p=0.749; p=0.403 |
ACS, acute coronary syndrome; ADHERE, Acute Decompensated Heart Failure Registry; AF, atrial fibrillation; AH, Adventist Health Off-the-shelf model; AMI, acute myocardial infarction; CABG, coronary artery bypass grafting; Car, cardiac-related; CHADS2, Congestive heart failure, Hypertension, Age, Diabetes, previous Stroke/transient ischemic attack; CHADS2-VASc, congestive heart failure, hypertension, age ≥ 75 years, diabetes mellitus, stroke or transient ischemic attack (TIA), vascular disease, age 65 to 74 years, sex category; CMS, Centers for Medicare and Medicaid Services; CRSS, CABG Readmission Risk Score; d, days; dev, development; FIM, motor and cognitive Functional Independence Measure; GRACE, Global Registry of Acute Coronary Events; HF, heart failure; HFpEF, heart failure with preserved ejection fraction; HLT, Hosmer-Lemeshow test; HOSPITAL, Hemoglobin level, discharged from Oncology, Sodium level, Procedure during admission, Index admission Type, Admission, Length of stay; HVD, heart valve disease; ICD, implantable cardioverter defibrillator; LACE, Length of stay, acuity of the Admission, Comorbidity of the patient and Emergency department use in the duration of 6 months before admission; NA, not applicable; NR, not reported; O, E, observed, expected; PARR-30, Patients at Risk of Re-admission within 30 days; PCI, percutaneous coronary intervention; plot, calibration plot; READMITS, Renal Function, Elevated Brain Natriuretic Peptide, Age, Diabetes Mellitus, Nonmale Sex, Intervention with Timely Percutaneous Coronary Intervention, and Low Systolic Blood Pressure; SILVER-AMI, Comprehensive Evaluation of Risk Factors in Older Patients with AMI; STS, Society of Thoracic; STS, Society of Thoracic; STS PROM, Society of Thoracic Surgeons Predicted Risk of Mortality; TAVR, transcatheter aortic valve replacement; val, validation.