Table 2.
Model performance characteristics for PRESENT-SHD for detection of structural heart disease across the held-out test set and external validation cohorts.
| Cohort Type | Site Name | Total Number | Diagnostic OR | AUROC | AUPRC | F1 Score | Prevalence | Sensitivity | Specificity | PPV | NPV |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Held-out test set | Yale New Haven Hospital | 6203 | 17.2 (14.7–20.1) | 0.886 (0.877–0.894) | 0.807 (0.791–0.823) | 0.7 | 33.60% | 89.8% (89.0–90.5) | 66.2% (65.0–67.4) | 57.4% (56.1–58.6) | 92.8% (92.1–93.4) |
| External validation – Hospital sites | Bridgeport Hospital | 8944 | 14.8 (12.9–16.9) | 0.854 (0.847–0.862) | 0.834 (0.823–0.845) | 0.751 | 46.60% | 93.2% (92.6–93.7) | 52.0% (51.0–53.1) | 62.9% (61.9–63.9) | 89.7% (89.1–90.3) |
| Greenwich Hospital | 2271 | 30.6 (22.2–42.1) | 0.900 (0.888–0.913) | 0.894 (0.878–0.910) | 0.798 | 49.80% | 96.0% (95.2–96.8) | 55.9% (53.9–58.0) | 68.3% (66.4–70.2) | 93.4% (92.4–94.4) | |
|
Lawrence + Memorial
Hospital |
11447 | 16.0 (14.0–18.2) | 0.871 (0.864–0.878) | 0.771 (0.757–0.784) | 0.643 | 31.50% | 92.5% (92.0–93.0) | 56.4% (55.5–57.3) | 49.3% (48.4–50.3) | 94.3% (93.8–94.7) | |
| Westerly Hospital | 1843 | 19.9 (14.5–27.3) | 0.887 (0.874–0.902) | 0.906 (0.890–0.922) | 0.81 | 55.60% | 95.1% (94.1–96.1) | 50.5% (48.3–52.8) | 70.6% (68.6–72.7) | 89.2% (87.8–90.6) | |
| External validation – Population-based cohort | ELSA-Brasil | 2988 | 11.4 (6.0–21.5) | 0.853 (0.811–0.897) | 0.354 (0.253–0.460) | 0.121 | 2.90% | 87.5% (86.3–88.7) | 61.9% (60.2–63.6) | 6.5% (5.6–7.4) | 99.4% (99.1–99.7) |
Abbreviations: AUPRC, area under the precision recall curve; AUROC, area under the receiver operating characteristic curve; NPV, negative predictive value; PPV, positive predictive value.